in reply to MCE segmentation fault
Hi Anonymous Monk,
Unfortunately, one cannot serialize Imager objects or will crash. This is true for Storable and Sereal::Encoder/Decoder. The thing to do is to save the image into a scalar variable before sending to the manager process.
Testing was done using Ubuntu 18.04.
$ sudo apt update $ sudo apt install libimager-perl libmce-perl $ sudo apt install libsereal-encoder-perl libsereal-decoder-perl $ sudo apt install ttf-mscorefonts-installer
MCE::Map demonstration
Here is the working parallel demonstration via MCE::Map.#!/usr/bin/perl use strict; use warnings; use Imager; use MCE::Map; STDOUT->autoflush; MCE::Map->init( max_workers => 6, chunk_size => 1, init_relay => '' ); my @data = mce_map { my $x = $_; my $i = Imager->new(xsize=>120, ysize=>50) or die Imager->errstr; $i->string( text => $x, color => Imager::Color->new('ffffff'), font => Imager::Font->new( file => '/usr/share/fonts/truetype/msttcorefonts/cour.ttf', # file => '/System/Library/Fonts/Courier.dfont', # face => 'Courier New', # mswin size => 42, aa => 1), x => 5, y => 35 ); # One cannot serialize Imager objects or will crash. # Instead save the image to a scalar and send that. # The manager process later reads from scalar refs. $i->write(data => \my $data, type => 'gif'); MCE::relay { print "\r$x" }; $data; } [ 0 .. 9999 ]; MCE::Map->finish; Imager->write_multi({ file => 'gif.gif', type => 'gif', gif_loop => 0, gif_delay => 1 }, map { Imager->read_multi(data => \$_) } @data) or die Imager->errst +r; print " frame GIF done!\n";
MCE::Loop demonstration
Here is another way using MCE::Loop. I call MCE gather inside the relay block.#!/usr/bin/perl use strict; use warnings; use Imager; use MCE::Loop; STDOUT->autoflush; my @data; MCE::Loop->init( max_workers => 6, chunk_size => 1, init_relay => '', gather => sub { push @data, $_[1]; print "\r$_[0]"; } ); mce_loop { my $x = $_; my $i = Imager->new(xsize=>120, ysize=>50) or die Imager->errstr; $i->string( text => $x, color => Imager::Color->new('ffffff'), font => Imager::Font->new( file => '/usr/share/fonts/truetype/msttcorefonts/cour.ttf', # file => '/System/Library/Fonts/Courier.dfont', # face => 'Courier New', # mswin size => 42, aa => 1), x => 5, y => 35 ); # One cannot serialize Imager objects or will crash. # Instead save the image to a scalar and send that. # The manager process later reads from scalar refs. $i->write(data => \my $data, type => 'gif'); MCE::relay { MCE->gather($x, $data) }; } [ 0 .. 9999 ]; MCE::Loop->finish; Imager->write_multi({ file => 'gif.gif', type => 'gif', gif_loop => 0, gif_delay => 1 }, map { Imager->read_multi(data => \$_) } @data) or die Imager->errst +r; print " frame GIF done!\n";
Chunking demonstration for HEDT systems
Reducing IPC overhead (i.e. relay) is possible simply by chunking.#!/usr/bin/perl use strict; use warnings; use Imager; use MCE::Loop; STDOUT->autoflush; my $count = 0; my @data; MCE::Loop->init( max_workers => MCE::Util::get_ncpu(), chunk_size => 100, init_relay => '', gather => sub { if (@_ == 1) { print "\r", $count++; } else { push @data, @{ $_[1] }; } } ); mce_loop { my ($mce, $chunk_ref, $chunk_id) = @_; my @i_data; for my $x (@{ $chunk_ref }) { my $i = Imager->new(xsize=>120, ysize=>50) or die Imager->errstr; $i->string( text => $x, color => Imager::Color->new('ffffff'), font => Imager::Font->new( file => '/usr/share/fonts/truetype/msttcorefonts/cour.ttf', # file => '/System/Library/Fonts/Courier.dfont', # face => 'Courier New', # mswin size => 42, aa => 1), x => 5, y => 35 ); # One cannot serialize Imager objects or will crash. # Instead save the image to a scalar and send that. # The manager process later reads from scalar refs. $i->write(data => \my $data, type => 'gif'); push @i_data, $data; MCE->gather($x); } MCE::relay { MCE->gather($chunk_id, \@i_data) }; } [ 0 .. 9999 ]; MCE::Loop->finish; Imager->write_multi({ file => 'gif.gif', type => 'gif', gif_loop => 0, gif_delay => 1 }, map { Imager->read_multi(data => \$_) } @data) or die Imager->errst +r; print " frame GIF done!\n";
This has been rather interesting. There is an overall speedup, but not what I expected. Well, here are the various ways to run parallel. One may call gather multiple times inside a MCE code block. However, MCE::relay must be one time only. Not more or less.
Regards, Mario
|
---|
Replies are listed 'Best First'. | |
---|---|
Re^2: MCE segmentation fault
by marioroy (Prior) on Mar 03, 2020 at 06:59 UTC | |
Re^2: MCE segmentation fault
by Anonymous Monk on Mar 03, 2020 at 07:25 UTC | |
by Anonymous Monk on Mar 03, 2020 at 07:58 UTC | |
by marioroy (Prior) on Mar 03, 2020 at 08:06 UTC | |
by vr (Curate) on Mar 04, 2020 at 01:57 UTC | |
by marioroy (Prior) on Mar 04, 2020 at 04:36 UTC | |
|
In Section
Seekers of Perl Wisdom