|Keep It Simple, Stupid|
Re^3: How to check lines that start with the same word then delete one of themby roboticus (Chancellor)
|on Apr 10, 2020 at 12:53 UTC||Need Help??|
You can use a hash to help you decide what to do on later lines something like this:
As you process your file, you record the important decisions you've made in the hash to help guide future decisions.
In the example I cobbled together, I used three rules:
Using these rules, when we run the program we get:
As you can see, we're able to handle all the rules with a single pass over the file with the help of a little bookkeeping.
As you've guessed in your original post, the nested loop can consume quite a bit of time for a large file. So it's worthwhile to think of ways you can do your processing without having to repeatedly scan the file.
What if you wanted to keep the *last* line starting with each key? One way would be to leave the logic the same, but to process the records in reverse order. Another way would be to change the way you handle the "seen" hash: Instead of checking whether you've processed the key or not, you could store the data you want to keep in it. That way, you can simply overwrite each record with a later record if you want, and then output them at the end. If you're keeping your data in memory, you can even come up with a method to process the data in *one* order and output the data in a *different* order to make your task simpler.
It's often a mistake to immediately jump in and solve the problem until you think about how to simplify things. Sometimes you'll find that a problem could easily be solved if the data came in a more convenient form or order. In those cases, it may be profitable to simply reshape or reorder the data to suit and then solve the simpler problem.
When your only tool is a hammer, all problems look like your thumb.