Re: Benchmarks aren't everything

Everytime I've looked at that regex ive thought the same thing: what a strange regex to use for a comparison benchmark.

One optimisation that is missing from pre 5.9 Perl is that there is nothing to convert /x|y|z/ into the far more efficient /[xyz]/.* If javas regex engine does contain a compile time optimisation of this sort its almost guaranteed to run faster than released version of Perl. On 5.9 and later it would be handled somewhat better as a TRIE which would be much faster than before, but still not as efficient as if it had been a char class. (Any C programmers out there interested in a neat hack, converting TRIE regops with only one row in their table to proper char class nodes would be cool).

This factor coupled with the fact that he doesnt actually show what strings he matched against make me somewhat dubious of the value of his assertions.

Without looking at the optimisations built into the Java regex engine (which no doubt they do not release) its hard to say why Perl would be slower than Java. I personally wouldnt necessarily assume that it was the backtracking logic that was at fault without a better understanding of the test cases that he used for his timings.

* Update: IOW its probably not fair to penalize a system for performing one task inefficiently when there is an equivelent construct that is efficient, unless in the context of an overall comparison.

---
$world=~s/war/peace/g

Comment on Re: Benchmarks aren't everything Select or Download Code

Replies are listed 'Best First'.

Re^2: Benchmarks aren't everything (rewritten regex is 290% faster than original)
by demerphq (Chancellor) on Oct 25, 2005 at 08:16 UTC

As a follow up here is a benchmark of the regex as posted against the equivelent regex that uses classes and not alternation, note the drammatic difference in run time. ($rex1 is the regex without alternations, $rex2 is the regex as posted, with newlines inserted to make it moderately easier to read)

Read more... (2 kB)

which has the following output:

Benchmark: running rex1, rex2 for at least 1 CPU seconds...
      rex1:  1 wallclock secs ( 1.16 usr +  0.00 sys =  1.16 CPU) @ 51
+.04/s (n=59)
      rex2:  1 wallclock secs ( 1.08 usr +  0.00 sys =  1.08 CPU) @ 12
+.99/s (n=14)
       Rate rex2 rex1
rex2 13.0/s   -- -74%
rex1 50.6/s 290%   --
6291
6291
[download]

So by removing the alternation we see a 290% speedup. What was the run time difference in the benchmark being referenced? Well apparently Java took 244 seconds and Perl took 527, which afaict (my maths arent so hot) is %115 faster. Now perhaps Java also would benefit from this modest optimisation of his code, but perhaps not, perhaps the Java implementation does this internally and in fact would not see an improvement. Either way the case does not seem to be proved at all.

BTW, for my benchmarks i just used the source for a page of Perlmonks. Use whatever XML/HTML you like.

---
$world=~s/war/peace/g

[reply]
[d/l]
[select]


go ahead... be a heretic
	PerlMonks