Hi, cmk
Very cool! Seeing my old parallel code made me want to try again for a 100% pure Perl solution not involving Inline::C. So first serial and parallel code afterwards. Workers consume about 16 MB each. The parallel code runs 3 times faster compared to serial. This is made possible by having workers write to STDOUT directly.
$ time perl serial.pl >/dev/null
$ time perl parallel.pl >/dev/null
Serial Demonstration
use strict;
use warnings;
my @vals = qw( 0.0 0.2 0.4 0.6 0.8 1.0 );
sub proc {
for my $a ( @vals ) {
for my $b ( @vals ) {
for my $c ( @vals ) {
for my $d ( @vals ) {
for my $e ( @vals ) {
for my $f ( @vals ) {
for my $g ( @vals ) {
for my $h ( @vals ) {
for my $i ( @vals ) {
for my $j ( @vals ) {
for my $k ( @vals ) {
print "$a\t$b\t$c\t$d\t$e\t$f\t$g\t$h\t$i\t$j\t$k\t1\t1\n";
}}}}}}}}}}}
}
proc();
Parallel Demonstration
use strict;
use warnings;
use MCE;
my @vals = qw( 0.0 0.2 0.4 0.6 0.8 1.0 );
# Must autoflush because workers write to STDOUT directly.
STDOUT->autoflush(1);
sub proc {
my $mce = MCE->new(
max_workers => scalar(@vals),
chunk_size => 1,
init_relay => 1,
user_func => sub {
my ($a, $b, $c) = @{ MCE->user_args };
my ($buf, $d ) = ( '', $_ ); # $d is the input
for my $e ( @vals ) {
for my $f ( @vals ) {
for my $g ( @vals ) {
for my $h ( @vals ) {
for my $i ( @vals ) {
for my $j ( @vals ) {
for my $k ( @vals ) {
$buf .= "$a\t$b\t$c\t$d\t$e\t$f\t$g\t$h\t$i\t$j\t$k\t1\t1\n";
}}}}}}}
# Relay is driven by the chunk_id value behind the scene.
# The benefit is orderly output, one worker at a time.
MCE::relay { print $buf };
}
)->spawn;
for my $a ( @vals ) {
for my $b ( @vals ) {
for my $c ( @vals ) {
# MCE workers persist between each run. The user_args option
# is how to pass parameters to them.
$mce->process({
input_data => \@vals,
user_args => [ $a, $b, $c ],
});
}
}
}
$mce->shutdown;
}
proc();
Parallel happens at the 4th level to minimize memory consumption.
Regards, Mario