Re: Re: Benchmark Arena Proposal

A very interesting link indeed! While Perl might not look to good in many individual tests, the scorecard page on that site allows assignment of weights to the benchmarks and under the default weights Perl does fairly well (I'll have to go back when I have more active brain cells and refine the weights a bit).

Now that other languages have been raised in this thread, I'd like to add the point that (for me at least) one major benefit of Perl is greatly reduced development time - in the dark ages I did all my text processing in Fortran or C (funny, they don't include Fortran on that site ;-). Now I imagine this would be very hard to do, but has anyone ever attempted to compare the development times? Sure it would be hard to avoid just comparing the coding proficiency of people, and stats on LoC per day I've seen have usually been pretty indefensible. How about errors per day? Errors per line? Define a line or error! One might get around all these silly measures by simply comparing total time to meet an end-product specification, especially since many fields use (or are moving to) an end-product focus. Of course that ignores maintainability...

All the same, I imagine someone (in some hairy-titled ACM journal perhaps) has looked at this question, and since it's an issue that Perl responds to well, it might be worth further investigation, vis-a-vis Perl advocacy.

In any case, while I do find Perl vs. Perl benchmarks most amusing, all a benchmark will tell you is how fast a given platform can run the benchmark!

--
I'd like to be able to assign to an luser

Comment on Re: Re: Benchmark Arena Proposal

Replies are listed 'Best First'.
Re: Re: Re: Benchmark Arena Proposal by salvadors (Pilgrim) on Jan 31, 2001 at 19:53 UTC
In any case, while I do find Perl vs. Perl benchmarks most amusing, all a benchmark will tell you is how fast a given platform can run the benchmark! I disagree. A good benchmark should compare 2 or more things, not to see how fast either runs, but to see how they run relative to each other. In this case Doug compares Perl to lots of other scripting languages - all on the same platform, and where possible, under the same conditions. The results are valid for seeing a lot more than how fast (in raw speed) something is. Tony	[reply]
Re: Re: Re: Re: Benchmark Arena Proposal by Albannach (Monsignor) on Jan 31, 2001 at 20:25 UTC
Oh, a good benchmark? ;-) OK, I'll admit I was being a bit flippant there. I'll agree with you if you add that the test must use real world data and real conditions as much as possible, but I just don't know how often that level of effort is made. In this case Doug compares Perl to lots of other scripting languages - all on the same platform, and where possible, under the same conditions. The results are valid for seeing a lot more than how fast (in raw speed) something is. I did read the page and know what he's comparing, but I still don't see what broader insights can be gained from these statistics. I'm not saying they are not interesting, but I sure wouldn't pick a language based on these results alone. Can you elaborate a bit on what you think the results show? If you compare languages or platforms using the more traditional benchmarks which are often highly idealized and optimized little exercises that have little to do with a real world use, you'll be able to say which ones run the benchmark the fastest, but not which one is better for a particular application (unless you have thoroughly profiled that application and know the appropriate weights to insert in the scorecard page or equivalent). -- I'd like to be able to assign to an luser	[reply]


Your skill will accomplish what the force of many cannot
	PerlMonks