Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Re: When Test Suites Attack

by Anonymous Monk
on Oct 28, 2005 at 22:09 UTC ( #503789=note: print w/replies, xml ) Need Help??


in reply to When Test Suites Attack

I've been finding more and more that small changes in my code are making huge changes in the output and trying to continuously update the tests to exactly match...
I don't know your problem domain, but would it be possible to test for other (higher level) invariants besides an exact textual match? For instance, maybe you could just check that a transformation preserves syntatic correctness, or that all tags are balanced, or properly nested, or conserved, etc. Let's take a different example. If you are creating a ray tracer, you could have a unit test that renders a red sphere on a black background and then compares that bitmap to a correctly rendered red sphere bitmap, checking that each pixel is correct. Or, you could instead write tests like...
  • Render a red sphere. Are any pixels in the bitmap red?
  • Render a red sphere with a large radius. Are there more red pixels than with a small radius?
  • Render a red sphere that is far away. Does it produce fewer red pixels than one that is close to the camera?
  • etc.
Does that make any sense?

Replies are listed 'Best First'.
Re^2: When Test Suites Attack
by Ovid (Cardinal) on Oct 28, 2005 at 22:32 UTC

    I'm taking a roughly similar strategy now. Previously I had many of these:

    is_xml $result, $expected, '... and we should receive the correct XML';

    By generating $expected with a different algorithm (to ensure that I'm not just reaching into buggy code), I had to constantly maintain two different sets of code which did the same thing. It was very painful. Now I'm doing this:

    is_well_formed_xml $result, '... and we should receive well-formed XML';

    And to verify that it's correct, I'm adding more high-level integration testing. It's an annoying trade-off, but like what you're suggesting, it's a reasonable one.

    Cheers,
    Ovid

    New address of my CGI Course.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://503789]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (3)
As of 2021-02-28 04:05 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?