Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

MCE::Channels 1.878 adds fast channel implementations

by marioroy (Prior)
on Feb 21, 2022 at 13:02 UTC ( [id://11141536]=CUFP: print w/replies, xml ) Need Help??

Greetings,

MCE::Channels has been there for some time. But missing were fast implementations without involving serialization i.e. non-Unicode strings. For implementations that involve serialization, MCE::Channel uses Sereal::Encoder and Sereal::Decoder if available. Otherwise, defaults to Storable for handling serialization.

How this came about is that someone wrote me and asked what does MCE provide for low-latency IPC communication. I replied nothing because involving serialization. So I took the existing implementations and removed the bits involving serialization, added Fast suffix to the name, and added corresponding test files.

I have been wanting to compare them all. Folks are not likely to notice a difference between a second or two for a long running application.

Below, channel implementations Threads and Mutex involve locking and serialization. ThreadsFast and MutexFast are similar but without serialization i.e. non-Unicode strings only. The Simple implementations lack locking supporting one worker on either end of the channel.

Threads

use strict; use warnings; use threads; use MCE::Channel; #my $chnl = MCE::Channel->new(impl => "ThreadsFast"); # 1.734s none #my $chnl = MCE::Channel->new(impl => "Threads"); # 2.247s Sereal # 3.232s Storab +le my $chnl = MCE::Channel->new(impl => "SimpleFast"); # 0.965s none #my $chnl = MCE::Channel->new(impl => "Simple"); # 1.940s Sereal # 3.305s Storab +le my $size = 1_000_000; my $thrd = threads->create(sub { my $ret; $ret = $chnl->recv() for 1..$size; }); $chnl->send("this is something $_") for 1..$size; $thrd->join();

Child process

use strict; use warnings; use MCE::Child; use MCE::Channel; #my $chnl = MCE::Channel->new(impl => "MutexFast"); # 3.552s none #my $chnl = MCE::Channel->new(impl => "Mutex"); # 4.025s Sereal # 4.815s Storab +le my $chnl = MCE::Channel->new(impl => "SimpleFast"); # 0.949s none #my $chnl = MCE::Channel->new(impl => "Simple"); # 1.644s Sereal # 3.286s Storab +le my $size = 1_000_000; my $proc = MCE::Child->create(sub { my $ret; $ret = $chnl->recv() for 1..$size; }); $chnl->send("this is something $_") for 1..$size; $proc->join();

Pretty much everything in MCE involves serialization. That is numbers remain numbers and not converted to a string. Likewise, Unicode strings and data structures are preserved as well. The fast channel implementations fill a void when serialization is not required.

Okay, this is nothing major. But I needed to let folks know.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: CUFP [id://11141536]
Approved by hippo
Front-paged by Discipulus
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chanting in the Monastery: (5)
As of 2024-04-18 06:53 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found