Beefy Boxes and Bandwidth Generously Provided by pair Networks
Come for the quick hacks, stay for the epiphanies.
 
PerlMonks  

Re^2: Code brewing for the upcoming MCE 10 year anniversary

by marioroy (Prior)
on Nov 04, 2022 at 06:53 UTC ( [id://11147959]=note: print w/replies, xml ) Need Help??


in reply to Re: Code brewing for the upcoming MCE 10 year anniversary
in thread Code brewing for the upcoming MCE 10 year anniversary

I have always wanted a Barrier module to complement MCE::Child, MCE::Hobo, or threads. The module works on Unix, Cygwin, including Windows. Tonight, I enhanced MCE::Simple to support the spawn loop syntax.

# create the worker processes spawn task($barrier, $_) for 1..5;

Which is transposed to the following. The ident argument defaults to blank '' if omitted.

MCE::Simple::_reap_joinable(); do { MCE::Simple::_async_i('', \&task, $barrier, $_) } for 1..5;

At this moment, the code is 100% completed in MCE::Barrier, MCE::Semaphore, and MCE::Simple. It's a relief. I tested MCE::Barrier on Linux with 4,000 workers, trying to break it :). All the one's come first, the two's, followed by three's and so on. Workers wait for all parties to reach the barrier.

use strict; use warnings; use MCE; use MCE::Barrier; use Time::HiRes qw(time); my $num_workers = 4000; my $barrier = MCE::Barrier->new($num_workers); my $start = time(); MCE->new( max_workers => $num_workers, user_func => sub { my $id = MCE->wid; for (1..400) { MCE->say("$_: $id"); $barrier->wait; } } )->run(); printf {*STDERR} "\nduration: %0.3f\n\n", time() - $start;

The sync implementation in MCE is two stages; sync and unsync. MCE::Barrier's implementation is one stage with alternating semaphores (1 or 2) k = 3 - k. A limiter prevents many workers attempting read IO concurrently or simultaneously. The result is a fast implementation no matter the number of parties.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11147959]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others about the Monastery: (2)
As of 2024-04-20 05:37 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found