Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

Re: Parallel::ForkManager right approach

by roboticus (Chancellor)
on Oct 13, 2019 at 17:59 UTC ( #11107401=note: print w/replies, xml ) Need Help??


in reply to Parallel::ForkManager right approach

Takamoto:

To test your code, I added this to the end:

sub tprint { my $t = time; print "$t: ", shift, "\n"; } sub getResultsAPI { my ($name, $doze_time) = @_; tprint "API_$name: dozing for ${doze_time}s"; sleep $doze_time; tprint "API_$name: done"; } sub getResultsAPI_1 { getResultsAPI(1, 12) } sub getResultsAPI_2 { getResultsAPI(2, 22) } sub getResultsAPI_3 { getResultsAPI(3, 16) } sub getResultsAPI_4 { getResultsAPI(4, 5) } sub getResultsAPI_5 { getResultsAPI(5, 11) } sub getResultsAPI_6 { getResultsAPI(6, 7) }

And it seemed to do what you'd expect:

$ perl pm_11107400.pl 1570989189: API_1: dozing for 12s 1570989189: API_2: dozing for 22s 1570989189: API_3: dozing for 16s 1570989189: API_4: dozing for 5s 1570989189: API_5: dozing for 11s 1570989194: API_4: done 1570989200: API_5: done 1570989201: API_1: done 1570989205: API_3: done 1570989211: API_2: done

Of course, if your APIs contend with each other for resources (such as hitting the hard drive a lot), you may not save much time. If they run well in parallel as the simple sleep timers do in the case I tried, you can save much more. Make sure you look at what your different APIs do to see if they conflict with each other. Bundling IO-heavy operations with CPU-heavy operations tends to work well to save you time. Similarly, you can *lose* time if you have processes always contending for the same resources.

Note that you have a minor bug, though, in that you never run API_6. I'm assuming the bug was introduced by your simplification of the test code.

...roboticus

When your only tool is a hammer, all problems look like your thumb.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://11107401]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (5)
As of 2020-10-22 21:53 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My favourite web site is:












    Results (230 votes). Check out past polls.

    Notices?