Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?

Re: Re: Re: Compiling Regular Expressions

by BrowserUk (Patriarch)
on Feb 19, 2003 at 10:21 UTC ( #236592=note: print w/replies, xml ) Need Help??

in reply to Re: Re: Compiling Regular Expressions
in thread Compiling Regular Expressions

I think your interpretation is correct, though when I've benchmarked it, now and previously, I have had variable results as you'll see below.

It seems to consistantly slow the regex down (even excluding the study time), if you only match against a single regex.

There can be some considerable speed up when matching against multiple regexes, but it's not consistant in how much you get. In the example below, matching against 2 regexes on a study'd string sometimes shows upto 50% improvement relative to an unstudy'd one, but less so when matching 3 regexes against the same two strings. It seems to be a function of the constant content of the regexs. Ie. what characters are involved. Those containing rare characters showing greater benefit than those not, but that's really educated speculation.

That said, there always seems to be some benefit to studying the string if you intend to match against more than once with 2 or more different regexes.

There doesn't seem to be any benefit (actually normally a small penalty) for multiple matches with a single regex, as with the /g modifier.

perl> use Benchmark qw[cmpthese]; perl> $s = "I say it's the opposite, because the FAQ says it takes mo +re time, and that it works on the scalar to be matched rather than the regex, but I might as well ask. In what circumstances would I + study() a string before hitting it with a regex?" perl> $t = $s perl> study $s perl> cmpthese( -1, { studied=>'$_=()=$s=~m[FAQ]o', slacker=>'$_=()=$ +t=~m[FAQ]o' } ) Benchmark: running slacker, studied, each for at least 1 CPU seconds. +.. slacker: 1 wallclock secs ( 1.01 usr + 0.00 sys = 1.01 CPU) @ 56 +776.24/s (n=57344) studied: 2 wallclock secs ( 1.12 usr + 0.00 sys = 1.12 CPU) @ 54 +807.31/s (n=61439) Rate studied slacker studied 54807/s -- -3% slacker 56776/s 4% -- perl> cmpthese( -1,{\ studied=>'$a = ($s = ~m[FAQ]o && $s =~ m[regex]o)',\ slacker=>'$b = ($t =~ m[FAQ]o && $t =~ m[regex]o)',\ }) Benchmark: running slacker, studied, each for at least 1 CPU seconds. +.. slacker: 2 wallclock secs ( 1.08 usr + 0.00 sys = 1.08 CPU) @ 98 +754.16/s (n=106852) studied: 2 wallclock secs ( 1.20 usr + 0.00 sys = 1.20 CPU) @ 65 +054.91/s (n=78196) Rate studied slacker studied 65055/s -- -34% slacker 98754/s 52% -- perl> cmpthese( -1,{\ studied=>'$a = ($s =~ m[FAQ]o && $s =~ m[regex]o) && $s =~ m[circumsta +nces]o',\ slacker=>'$b = ($t =~ m[FAQ]o && $t =~ m[regex]o) && $t =~ m[circumsta +nces]o',\ }) Benchmark: running slacker, studied, each for at least 1 CPU seconds. +.. slacker: 2 wallclock secs ( 1.00 usr + 0.00 sys = 1.00 CPU) @ 71 +679.00/s (n=71679) studied: 2 wallclock secs ( 1.17 usr + 0.00 sys = 1.17 CPU) @ 56 +503.84/s (n=66166) Rate studied slacker studied 56504/s -- -21% slacker 71679/s 27% -- perl> cmpthese( -10,{\ studied=>'$a = ($s =~ m[FAQ]o && $s =~ m[regex]o)',\ slacker=>'$b = ($t =~ m[FAQ]o && $t =~ m[regex]o)',\ }) Benchmark: running slacker, studied, each for at least 10 CPU second +s... slacker: 10 wallclock secs (11.21 usr + 0.01 sys = 11.22 CPU) @ 1 +11296.69/s (n=1248415) studied: 11 wallclock secs (10.00 usr + 0.00 sys = 10.00 CPU) @ 1 +41637.98/s (n=1417088) Rate slacker studied slacker 111297/s -- -21% studied 141638/s 27% --

Examine what is said, not who speaks.

The 7th Rule of perl club is -- pearl clubs are easily damaged. Use a diamond club instead.

Replies are listed 'Best First'.
Re^4: Compiling Regular Expressions
by diotalevi (Canon) on Feb 19, 2003 at 14:32 UTC

    A reading of _Mastering Regular Expressions_ leaves me with the impression that sometimes study() speeds things up but other times it just slows things down. So its actual behaviour on performance is likely dependant on your data, your regexes (which is also another sort of data) and then how often each data+regex combination is used. The way M_R_E leaves things, I'm not even sure you can arrive at any sort of truth by using benchmarks. Or... you can arrive at a local definition of truth only by testing your specific data + your specific regexes together.

    The general idea is that a generalized benchmark for study() is invalid.

    Seeking Green geeks in Minnesota

      It should also be noted that study() is not as well tested in perl as it ought to be, and as a result changes to the regexp engine over the years have managed to introduce various problems that cause study() not to gain as much speed as it should, and in some cases even give the wrong answer.

      In particular, I don't think study() knows anything about Unicode (though I haven't checked).

      The plan is that 5.10.0 should involve a big cleanup of the regexp engine, so this may improve in the future.


Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://236592]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (2)
As of 2022-01-25 12:19 GMT
Find Nodes?
    Voting Booth?
    In 2022, my preferred method to securely store passwords is:

    Results (66 votes). Check out past polls.