Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re^2: Why this code is so slow if run in thread?

by vr (Curate)
on Dec 11, 2016 at 17:38 UTC ( #1177611=note: print w/replies, xml ) Need Help??


in reply to Re: Why this code is so slow if run in thread?
in thread Why this code is so slow if run in thread?

But then Perl's Core "Encode" is not thread safe? :(

Suppose I split my program in two. First one creates 2 files, with single-byte encoded data and double-byte encoded data (and says "Equal", of course -- this image has less than 255 "objects").

use strict; use warnings; use 5.020; use PDL; use PDL::IO::Image; use PDL::Image2D; use PDL::IO::FastRaw; use Encode qw/ decode /; my $img = PDL::IO::Image-> new_from_file( 'test.png' ); my $pdl = $img-> pixels_to_pdl-> short; my $s1 = cc8compt( $pdl == 0 )-> byte; my $s2 = cc8compt( $pdl == 0 ); say 'Equal' if decode( 'UTF16LE', ${ $s2-> get_dataref }) eq ${ $s1-> get_dataref }; writefraw( $s1, "fname-1" ); writefraw( $s2, "fname-2" );

The 2nd program doesn't use PDL modules, it reads data from disk (and uses some hard-coded numbers):

use strict; use warnings; use 5.020; use threads; use Encode qw/ decode /; say "No thread (byte)\n---------"; say 'Count: ', scalar @{ test( 1 )}; say "\nThread (byte)\n---------"; say 'Count: ', scalar @{ threads-> create( \&test, 1 )-> join }; say "\nNo thread (short)\n---------"; say 'Count: ', scalar @{ test( 2 )}; say "\nThread (short)\n---------"; say 'Count: ', scalar @{ threads-> create( \&test, 2 )-> join }; sub test { my $arg = shift; open my $fh, '<', "fname-$arg"; binmode $fh; my $str = do { local $/; <$fh> }; close $fh; $str = decode( 'UTF16LE', $str ) if $arg == 2; my ( $w, $h ) = ( 7616, 1200 ); my @b = map { [ [ $w, 0 ], [ $h, 0 ] ] } 0 .. 145; my $t = time; for my $y ( 0 .. $h - 1 ) { my $s = substr( $str, $y * $w, $w ); while( $s =~ m[[^\0]+]g ) { my $c = ord( $& ); $b[ $c ][ 0 ][ 0 ] = $-[0] if $-[0] < $b[ $c ][ 0 +][ 0 ]; $b[ $c ][ 0 ][ 1 ] = $+[0] - 1 if $+[0] - 1 > $b[ $c ][ 0 +][ 1 ]; $b[ $c ][ 1 ][ 0 ] = $y if $y < $b[ $c ][ 1 +][ 0 ]; $b[ $c ][ 1 ][ 1 ] = $y if $y > $b[ $c ][ 1 +][ 1 ]; } } say 'Time: ', time - $t; shift @b; return \@b; }

The output:

No thread (byte) --------- Time: 0 Count: 145 Thread (byte) --------- Time: 1 Count: 145 No thread (short) --------- Time: 1 Count: 145 Thread (short) --------- Time: 51 Count: 145

Replies are listed 'Best First'.
Re^3: Why this code is so slow if run in thread? (create)
by tye (Sage) on Dec 11, 2016 at 19:24 UTC

    With Perl threads, the thing that is usually the slowest is creating a thread. This is because creating a Perl thread requires cloning every data structure you have created. So I would first determine if that is where the time is being spent. You can report that by comparing the time at the start of the routine running against the time just before you ask for a thread to be created.

    If that is indeed where most of the time is spent, then you should do what just about anybody who has gotten good at using Perl threads does: Create your threads very early and just ship work to them, usually via a thread queue such as Thread::Queue.

    Though, looking at your code, I don't see where you would have a large data structure that should not have been destroyed before you create the thread. But, that data could be something cached (perhaps by accident) in some module you have pulled in, for example.

    Update: Or it could be time spent destroying the second interpreter instance.

    - tye        

Re^3: Why this code is so slow if run in thread?
by ikegami (Patriarch) on Dec 11, 2016 at 19:14 UTC

    Did you look at your results? They're mostly useless!

    We can tell that shorts are 50x slower with threads, but we have no idea how much slower bytes are with threads. Could be there not slower at all (0.9999s vs 1.0000s). Could be billions time slower (0.000000001 vs 1.000000000).

    Please re-run the tests with use Time::HiRes qw( sleep );. Or provide the data files.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1177611]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (2)
As of 2022-10-06 19:30 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My preferred way to holiday/vacation is:











    Results (27 votes). Check out past polls.

    Notices?