http://qs321.pair.com?node_id=323489

AcidHawk has asked for the wisdom of the Perl Monks concerning the following question:

I have started to realise that this is a HUGE subject, with poeple devoting much of their time to comparing languages in shootouts etc. for performance gains.

My requirement is not as vast. What got me started down this line is a real issue I am currently experiencing.

I have script that reads a dir for files, when files are found it connects to a server and moves these files over the socket to the server. When I (using a test script) put a file into the watched dir I notice that the memory for this client process(which has been compiled using perl2exe) increases but then stays pretty static. I can actually send about 100 files in a few seconds and still see no increase in memory utilisation. However when I send 200 files the momory jumps again. The problem is that it doesn't release this memory.

I am currently using Devel::Size to test which of my vars/arrays/hashes etc are growing, but have to date not found where my problem is. I am also looking at putting Benchmark::Timer to use.

This really leads me to my current musing...

  • Should I start building test/benchmark code into my future scripts for debugging purposes..?
  • What do other monks do for benchmarking/testing their scripts?
  • When do you reach the point of "OK, this has too many tests"?
  • What other metrics do you measure (I.e. speed, memory)?

    • -----
    Of all the things I've lost in my life, its my mind I miss the most.

    Replies are listed 'Best First'.
    Re: Benchmark testing
    by Abigail-II (Bishop) on Jan 23, 2004 at 10:57 UTC
      Some opinions:
      Should I start building test/benchmark code into my future scripts for debugging purposes..?
      As a general rule, no. It makes your code messy, and you'll do a lot of work that you may not use at all.
      What do other monks do for benchmarking/testing their scripts?
      print statements (sometimes in combination with YAML - never with Data::Dumper). make test. I seldomly benchmark programs I write, but I do benchmark theoretical cases.
      When do you reach the point of "OK, this has too many tests"?
      With about 150,000 tests in Regexp::Common, I say "never".
      What other metrics do you measure (I.e. speed, memory)?
      Several others, of which the most important is I/O. With modern hardware, I/O is often a bigger bottleneck than CPU speed or memory. But of course, it depends on the problem and what else is going on on the box.

      Abigail

    		[reply]
    Re: Benchmark testing
    by adrianh (Chancellor) on Jan 23, 2004 at 10:50 UTC
      When do you reach the point of "OK, this has too many tests"?

      My attitude to optimisation and benchmarking is neatly summarised by these two quotes:

      "Premature optimization is the root of all evil in programming."
      -- C.A.R. Hoare
      Rules of Optimisation:
      Rule 1: Don't do it.
      Rule 2 (for experts only): Don't do it yet.
      -- M.A. Jackson

      Basically, CPU cycles are cheap. It's much more important to write clear elegant code that you can come back to in six months and still understand.

      Most of the time you'll find that the most obvious clear code is more than fast enough. I only reach for Benchmark in the very few cases when this isn't the case.

    		[reply]
    Re: Benchmark testing
    by parkprimus (Sexton) on Jan 23, 2004 at 14:58 UTC
      There is an article in this months Linux Journal which talks about this very issue. Might be of some use!
    		[reply]
    Re: Benchmark testing
    by BrowserUk (Patriarch) on Jan 23, 2004 at 17:18 UTC

      If you are the only user of your code, and you only ever run your code stand-alone, and it completes it's task(s) within a timeframe that is reasonable, you will never have occasion to need to benchmark.

      However, sometimes your code will have to share a box with other code. Sometimes, throwing more hardware at a problem will not be an option. Sometimes the obvious, simple solution will not be quick enough for the requirements, and will leave people waiting for answers.

      Does anyone ever say: "I wish program X took twice as long to load and consumed 3 times as much memory"?

      Does any user of a program ever say: "I don't mind the bloat"; or "I don't mind waiting for the answers"?

      Or is it only programmers that say, tardy profligacy should be a way of life?

      Examine what is said, not who speaks.
      "Efficiency is intelligent laziness." -David Dunham
      "Think for yourself!" - Abigail
      Timing (and a little luck) are everything!
    		[reply]

    Back to Seekers of Perl Wisdom