To Fork or Not to Fork. Tis a question for PerlMonksby pimperator (Acolyte)
|on Jul 01, 2014 at 01:15 UTC
pimperator has asked for the wisdom of the Perl Monks concerning the following question:
Hello Monks, I have a question about forking processes using the ForkManager module. I'm running a script that will parse a large file, there are about 10,000 of these files. The initial script I wrote forks the process into 15 child processes that parses one of the 10,000 files. It took around 5 days for the code to finish (person running it pauses it sometimes). The code processed 1 file every 1-2 minutes. I'm thinking because ForkManager only creates new processes and does not 'really thread' that the processing time would be the same if I chose not to fork. Is this true? Basically, would it take the same amount of time to parse 10,000 files one by one compared to parsing 15 of them at the same time? I'm asking this and not testing it because I'm writing a code for someone else to run on a computer that I cannot access due to sensitive information. So... yeah Tusen Takk Monks.