Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re^8: Interrupt multi-process program while using MCE::Shared hash: END block code does not (all) run

by 1nickt (Canon)
on Apr 09, 2017 at 19:56 UTC ( [id://1187535]=note: print w/replies, xml ) Need Help??


in reply to Re^7: Interrupt multi-process program while using MCE::Shared hash: END block code does not (all) run
in thread Interrupt multi-process program while using MCE::Shared hash: END block code does not (all) run

Hi ++marioroy,

That's very cool. Now when one of the workers encounters a fatal exception the whole program ends.

But, of course, one has no guarantee of which tasks might have been completed (especially in a real-world scenario where task execution time varies and there are more than just two workers). So even with the program exiting via __DIE__, one can easily wind up with not only a partially populated hash/cache (as expected on early exit), but a hash partially populated *out of sequence* compared to the array being processed by mce_loop. E.g.:

Parent PID 21987 worker 2 (21990) processing chunk 1 worker 1 (21989) processing chunk 2 worker 1 (21989) processing chunk 4 worker 2 (21990) processing chunk 3 worker 1 (21989) processing chunk 6 worker 2 (21990) processing chunk 5 Illegal division by zero at mce12.pl line 35, <__ANONIO__> line 6. Hello from END block: 21990 ## mce12.pl: caught signal (__DIE__), exiting Hello from INT: 21989 Hello from END block: 21989 Hello from END block: 21987 Parent in END: $VAR1 = { '00 21990' => '1491767068', '01 21989' => '1491767068', '02 21990' => '1491767070', '03 21989' => '1491767070', '05 21989' => '1491767072' };
That was of course completely foreseeable, but I hadn't thought about it when asking for default DIE behaviour. Now, since it makes no difference in terms of the issue I first thought of (patchy-incomplete results), I think I may be more likely to favour the previous behaviour; in other words, continue processing even when one worker dies unexpectedly. Haha, sorry! Well, I think both choices are valuable and most needed, actually.

Thanks again.


The way forward always starts with a minimal test.
  • Comment on Re^8: Interrupt multi-process program while using MCE::Shared hash: END block code does not (all) run
  • Select or Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1187535]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chilling in the Monastery: (7)
As of 2024-04-18 09:57 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found