http://qs321.pair.com?node_id=496829


in reply to Re^4: too much testing?
in thread too much testing?

OK, so as to your question(s) - are you testing "too much"

I have strong opinions on this, based on years of writing code that has experienced jut about every failure mode imaginable.

I have a personal belief that you have tested enough when you feel you have exercised as much of the code as you reasonably can. I use Devel::Cover to tell me how much I haven't done, and use my experience and judgement to make informed decisions as to what tests I will add for the remains.

For example, do you need to test what your code does if read() fails ? If so, writing test cases to do this is a bit difficult, but if you need it, do it.
Otherwise, code and test on the assumption "read() will never fail" and move on.

Devel::Cover isn't perfect, and sometimes it can be very difficult to work with - for example, it complains about my $class = ref($proto)||$proto; as having an untested path in most "reasonable" usages. You have to write a test that calls the constructor in a manner you would expect to see for a function-based module e.g. Module::new(), not Module->new()
So you need to ask yourself "is testing for people calling my OO methods as normal functions something I want to do ?"

If it is important to you, you help newbies who do "Module::new" by mistake.

If not, ignore the report of the untested path from D::C and move on.

As an aside, I wrote a meditation on what I had to go through to get a reasonably large body of modules to get to 100% coverage as reported by D::C - see Lessons learned from getting to 100% with Devel::Cover about my struggles and discoveries from doing a deep testing effort.

...reality must take precedence over public relations, for nature cannot be fooled. - R P Feynmann

Replies are listed 'Best First'.
Re^6: too much testing?
by geektron (Curate) on Oct 03, 2005 at 08:04 UTC
    i do remember reading that meditation, and i'm going to re-read it in a few minutes.

    Devel::Cover is going to be my basis for "percentages". in this particular case, trying to achieve 100% coverage may not be practical at all, at least not pre-application release. (i say this *only* because these are single-site modules and not a public-consumption distro. i'm sure i'll add more features and tests later, and can try to come closer to 100% coverage.)

    in any case, thanks for some opinions on knowing when to say "enough is enough".