Your skill will accomplish what the force of many cannot |
|
PerlMonks |
IO::Async::PID works fine in SSCCE, fails randomly in real app, how to debug?by vr (Curate) |
on Jun 12, 2021 at 13:25 UTC ( [id://11133803]=perlquestion: print w/replies, xml ) | Need Help?? |
vr has asked for the wisdom of the Perl Monks concerning the following question: Edit. Please ignore. I forgot there's already "1 while waitpid(-1, WNOHANG) > 0;" added long time ago to obscure part of application to reap rarely spawned children, should re-write it now using e.g. a no-op IO::Async::PID's on_exit handler. Sorry. The OS is Windows, there's existing IO::Async-based, somewhat complex and clumsy app, which I want to extend. At some point in workflow, lists of file names arrive, in batches up to hundreds of items, but usually, say, 1-20. What I want to insert, is to process each file invoking external CLI tool, which takes, say, 5 to 60 seconds per file. It is to be done asynchronously by a few (2-5, I'll see when I get to observing real load) workers, and when whole current batch is ready, I want to get control back and proceed with workflow as it exists now. Because it's Windows, the recommended way with IO::Async::Process won't work, it wants real POSIX fork. "OK", I thought, I'll fire "system 1, @args" per file, and PID will be used to add IO::Async::PID watcher to the loop. For now, I haven't even got to using long-running CLI tool, I'm doing simple "echo filename" instead for experiments. Actually, in not-so-Small-SCCE, they are batches of numbers, not filenames. The code below runs fine as expected, indefinitely. But, when added almost "as-is" to real-life (RL) app, it fails at unpredictable moment, leaving me unpleasant task of cleaning the mess of interrupted workflow.
Because SSCCE works OK, I can only presume on_exit handler fails to be called, when computer is under heavy load (?) when RL app runs. External programs and other parts of RL app do much heavy lifting. How is it even possible, and why? How to simulate this load in SSCCE? How can I debug this? (storing futures as values in "active" hash was adopted from other parts of workflow, it serves no purpose in this SSCCE, please ignore)
and Runner.pm:
Back to
Seekers of Perl Wisdom
|
|