Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Re^3: About text file parsing

by marioroy (Prior)
on Aug 30, 2018 at 14:17 UTC ( [id://1221377]=note: print w/replies, xml ) Need Help??


in reply to Re^2: About text file parsing
in thread About text file parsing

Once again, hi :)

Using a simplified demonstration, regular expression appears to be 3x slower in Perl v5.20 and higher. I'm not sure why.

use strict; use warnings; use MCE; sub task { my ( $mce, $slurp_ref, $chunk_id ) = @_; # open file handle to scalar ref open my $input_fh, "<", $slurp_ref; while (<$input_fh>) { if (/^sample\s+(\S+)/) { ; } elsif (/^good\s+(\S+)/) { ; } } close $input_fh; } MCE->new( chunk_size => '1m', max_workers => 4, use_slurpio => 1, user_func => \&task ); MCE->process({ input_data => "test.txt" }); MCE->shutdown;

Results

$ time /opt/perl-5.8.9/bin/perl -I. test_demo.pl real 0m3.826s user 0m14.352s sys 0m0.133s $ time /opt/perl-5.10.1/bin/perl -I. test_demo.pl real 0m4.369s user 0m16.935s sys 0m0.126s $ time /opt/perl-5.12.5/bin/perl -I. test_demo.pl real 0m4.889s user 0m18.944s sys 0m0.134s $ time /opt/perl-5.14.4/bin/perl -I. test_demo.pl real 0m4.860s user 0m18.865s sys 0m0.127s $ time /opt/perl-5.16.3/bin/perl -I. test_demo.pl real 0m4.815s user 0m18.724s sys 0m0.129s $ time /opt/perl-5.18.4/bin/perl -I. test_demo.pl real 0m4.668s user 0m18.356s sys 0m0.116s $ time /opt/perl-5.20.3/bin/perl -I. test_demo.pl real 0m14.195s user 0m49.155s sys 0m7.282s $ time /opt/perl-5.22.4/bin/perl -I. test_demo.pl real 0m14.316s user 0m49.586s sys 0m7.041s $ time /opt/perl-5.24.3/bin/perl -I. test_demo.pl real 0m14.612s user 0m50.251s sys 0m7.531s $ time /opt/perl-5.26.1/bin/perl -I. test_demo.pl real 0m14.212s user 0m49.418s sys 0m6.999s $ time /opt/perl-5.28.0/bin/perl -I. test_demo.pl real 0m14.308s user 0m49.476s sys 0m7.137s

Regards, Mario

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1221377]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others musing on the Monastery: (3)
As of 2024-04-19 06:07 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found