Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

Re: How to speed up multiple regex in a loop for a big data?

by Samy_rio (Vicar)
on May 25, 2006 at 06:17 UTC ( [id://551517]=note: print w/replies, xml ) Need Help??


in reply to How to speed up multiple regex in a loop for a big data?

Hi, If I understood your question correctly, try this & see the Benchmark,

#!/usr/bin/perl use strict; use warnings; use Benchmark 'cmpthese'; cmpthese(-1, { method1 => '&yours', method2 => '&new_one', }); sub yours{ my %mapper; open(MAP, "<map.ini"); while (<MAP>) { chomp; tr/A-Z/a-z/; my @map_line = split (/\t/); $mapper{$map_line[0]} = $map_line[1]; } close(MAP); open(IN, "<input1.txt"); open(OUT, ">input_new.txt"); while (<IN>) { #print "%"; tr/A-Z/a-z/; foreach my $key (sort keys %mapper) { s/\b$key\b/$mapper{$key}/g; } print OUT "$_"; } close(IN); close(OUT); } sub new_one{ open(MAP, "map.ini"); my $map = do{local $/;<MAP>}; close(MAP); my %mapper; $map = lc($map); %mapper = map{split/\t/,$_} split(/\n/, $map); open(IN, "input1.txt"); my $input = do{local $/;<IN>}; close(IN); open(OUT, ">input_new.txt"); $input = lc($input); foreach my $key (sort keys %mapper) { $input =~ s/\b$key\b/$mapper{$key}/g; } print OUT $input; close(OUT); } __END__ Rate method1 method2 method1 908/s -- -13% method2 1050/s 16% --

I have check with less number of lines file only. You can check with your input.

Regards,
Velusamy R.


eval"print uc\"\\c$_\""for split'','j)@,/6%@0%2,`e@3!-9v2)/@|6%,53!-9@2~j';

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://551517]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others contemplating the Monastery: (4)
As of 2024-04-23 19:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found