Hi again perlygapes,

The MCE::Loop code just abstracts away all your Parallel::ForkManager logic and improves it, just as Parallel::ForkManager abstracts away and improves some of the tedious manual work of using fork() directly. See how the logic is encapsulated in a sub just like in your code, only with less concurrency boilerplate.

"using a separate DB connection instead for each child feels intuitively right"

I agree, the code I shared keeps a connection open for each child, which itself stays alive and handles multiple jobs from the job list as managed by MCE.

Here's a simpler example I've shared recently showing how to parallelize existing code for making a series of HTTP requests. How would you do the same using P::FM?

single process

use strict; use warnings; use 5.010;
use Data::Dumper; use HTTP::Tiny;
use Time::HiRes 'gettimeofday', 'tv_interval';

my $ua   = HTTP::Tiny->new( timeout => 10 );
my @urls = qw< gap.com amazon.com ebay.com lego.com wunderground.com
    imdb.com underarmour.com disney.com espn.com dailymail.com >;

my %report;

foreach( @urls) {
    my $start = [gettimeofday];
    $ua->get('https://' . $_);
    $report{$_} = tv_interval($start, [gettimeofday]) );
});

say Dumper \%report;
[download]

six processes
(workers stay alive, looping through the list, writing to a shared hash)
(one added line, two slightly changed lines)

use strict; use warnings; use 5.010;
use Data::Dumper; use HTTP::Tiny;
use Time::HiRes 'gettimeofday', 'tv_interval';
use MCE; use MCE::Shared;

my $ua   = HTTP::Tiny->new( timeout => 10 );
my @urls = qw< gap.com amazon.com ebay.com lego.com wunderground.com
    imdb.com underarmour.com disney.com espn.com dailymail.com >;

my $report = MCE::Shared->hash;

MCE->new( max_workers => 6 )->foreach( \@urls, sub {
    my $start = [gettimeofday];
    $ua->get('https://' . $_);
    $report->set( $_, tv_interval($start, [gettimeofday]) );
});

say Dumper $report->export;
[download]

Update: fixed error in first demo code, ++choroba

Hope this helps!