Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re^5: How to check lines that start with the same word then delete one of them

by hippo (Chancellor)
on Apr 10, 2020 at 13:15 UTC ( #11115318=note: print w/replies, xml ) Need Help??


in reply to Re^4: How to check lines that start with the same word then delete one of them
in thread How to check lines that start with the same word then delete one of them

The problem is that I've a big file (40000 line), that's why I did'nt bring it to my post.

Everyone is glad you didn't. "Sample data" means just that. Pick maybe 6 lines - enough to be illustrative and to cover the bases. My script above is a test only. It illustrates that the algorithm and the code works, given the sample data.

I just want to check if for every line there is another line witch have the same first word then I check it to delete one of them,

This is precisely what my test shows. Would you not agree? To turn the test into a working script just replace @in with the code you already have which reads the input data from the file and similarly write @have to your file at the end.

that's why I did two loops. but the algorithm is so slow

Your algorithm is O(n2) whereas mine is O(n). Mine should therefore be thousands of times faster for a 40,000 line dataset.

See also: Big O notation, SSCCE and Basic Testing Tutorial. HTH.

Replies are listed 'Best First'.
Re^6: How to check lines that start with the same word then delete one of them
by rsFalse (Hermit) on Apr 10, 2020 at 15:13 UTC
    I think your solution works in O(n*log n), because searching an item in a hash takes log n. Am I right? Still much much faster that O(n*n) :)
      No, a hash lookup is usually O(1), it does take some time, sure, but the time it takes does not depend (generally) on the hash size.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://11115318]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (7)
As of 2020-06-01 16:50 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Do you really want to know if there is extraterrestrial life?



    Results (5 votes). Check out past polls.

    Notices?