Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re^4: Reduce CPU utilization time in reading file using perl

by madtoperl (Hermit)
on Sep 30, 2013 at 06:39 UTC ( #1056290=note: print w/replies, xml ) Need Help??


in reply to Re^3: Reduce CPU utilization time in reading file using perl
in thread Reduce CPU utilization time in reading file using perl

Hi BrowserUk
Thanks a lot for the help.I have two huge files and need to compare the difference between those two files by column wise and write the difference of the corresponding column and row where the mismatch is into a third file. The lines of both the files are delimited using |.Could you please suggest the better option for this. Right now, I am loading the two files data into two separate hash and compare it and write it into the third file. It would be ood if you can suggest something other than loading the file content into database and fetching it.
Thanks,
madtoperl
  • Comment on Re^4: Reduce CPU utilization time in reading file using perl

Replies are listed 'Best First'.
Re^5: Reduce CPU utilization time in reading file using perl
by BrowserUk (Patriarch) on Sep 30, 2013 at 08:42 UTC
    It would be ood if you can suggest something other than loading the file content into database and fetching it.

    Probably, but not based on the information you've provided so far. Why do you seem reluctant to provide information?

    Please provide:

    • The size in bytes of both files.
    • The number of records in both files.
    • The number of fields in the lines of both files.
    • A couple of sample records from both files.

      If the data is proprietary, then take a couple of sample records and change the identifying words, numbers etc., but try to ensure that they remain realistic.

    • And idea of how often you will need to do this and how often the file(s) change.

      Ie. "The bigger file remains constant and the smaller changes one a week";

      Or: "This is a one-off problem never to be repeated".

      Or: "The two files never change, but the (combination of) fields used for comparison changes every day".

      Or: ...

    With that information, we here have a realistic chance of understanding the scale of the problem and possible solutions.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1056290]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others having an uproarious good time at the Monastery: (6)
As of 2023-12-06 17:20 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    What's your preferred 'use VERSION' for new CPAN modules in 2023?











    Results (31 votes). Check out past polls.

    Notices?