Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

No plan "weakens" my Test scripts?

by thor (Priest)
on Feb 08, 2005 at 05:29 UTC ( [id://428930]=perlquestion: print w/replies, xml ) Need Help??

thor has asked for the wisdom of the Perl Monks concerning the following question:

Fellow monks,

I was reading the POD for Test::More today and something struck me as odd:

There are rare cases when you will not know beforehand how many tests your script is going to run. In this case, you can declare that you have no plan. (Try to avoid using this as it weakens your test.)
Would someone mind explaining why that weakens the test? As a contrived example, what if I want to test that every record in a file got loaded to a database? The ok function (and it's bretheren) are very good at making assertion-style statements (I personally like "is"). I don't know how many records will be in the file, so I won't know how many tests will be executed before hand.

thor

Feel the white light, the light within
Be your own disciple, fan the sparks of will
For all of us waiting, your kingdom will come

Replies are listed 'Best First'.
Re: No plan "weakens" my Test scripts?
by saintmike (Vicar) on Feb 08, 2005 at 05:40 UTC
    Because you won't know if your test suite crashes right in the middle:
    use Test::More qw(no_plan); ok(1); # CRASH!! exit(0); ok(2);
    If you run that with the harness as in PERL_DL_NONLAZY=1 perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" test.pl it will happily report
    All tests successful. Files=1, Tests=1
    because it doesn't know better. If you had specified the desired outcome, as in
    use Test::More tests => 2;
    the test script would have reported
    # Looks like you planned 2 tests but only ran 1.
    and the harness would have failed, allowing you to catch the bug while it goes unnoticed otherwise.
Re: No plan "weakens" my Test scripts?
by chromatic (Archbishop) on Feb 08, 2005 at 05:43 UTC

    What if there's a problem in your test data and you end up loading ten times the number of records that you intended to run? How will you catch that?

    Suppose a query changes and accidentally pulls too much or too little data. How will you catch that?

    I feel a little more paranoid than usual any time my tests have loops or depend on data that comes from somewhere else. I really like having test plans there to catch my assumptions then.

      What if there's a problem in your test data and you end up loading ten times the number of records that you intended to run? How will you catch that?
      Because one of the tests is "count it on the database". If I ended up loading 10 times as much data, count on the database would be inflated and therefore not equal.

      I should be clear here, I'm not testing a module...I'm testing that a database conversion went as expected. In college, I had a warehouse job. Our mantra there was "If it came in by count, count it. If it came in by weight, weigh it". That's exactly what I'm doing here. I do counts before and after as a first level check. One of the tables is non-normalized in that it has the same 3 columns repeated over and over to reperesent different things (think a family; the columns are dad's hair color, occupation, and age, mom's hair color, occupation, and age, etc). There are a different number of sets depending on the database. For each of these sets of columns, I want to check that it got into the normalized structure correctly. Once the counts are verified, I "weigh" it. That is to say that I make sure that the quality of the data before and is the same.

      thor

      Feel the white light, the light within
      Be your own disciple, fan the sparks of will
      For all of us waiting, your kingdom will come

Re: No plan "weakens" my Test scripts?
by leriksen (Curate) on Feb 08, 2005 at 09:53 UTC
    I find Test::More qw(no_plan); incredibly convenient whilst I am developing the module and tests in question, but once I can say "Finished", then put in the number of tests. This way you get the ease of not having to annoyingly update the test number, but get the protect once your done.

    ...it is better to be approximately right than precisely wrong. - Warren Buffet

      This is a generally good approach, applicable to many tasks we programmers do. A "stub" for development that makes it easier and can be later removed when the system is released.

      It's important not to forget to remove the stub though !

      Actually, since I want to write my tests in an intentive manner, I never use no_plan. I want to be completely aware of how many tests I'm adding, because each set of tests should be testing a very specific scenario I have in mind. Otherwise, I'm not doing any better than slinging tests around. (This is equivalent to slinging code around.)

      Being right, does not endow the right to be rude; politeness costs nothing.
      Being unknowing, is not the same as being stupid.
      Expressing a contrary opinion, whether to the individual or the group, is more often a sign of deeper thought than of cantankerous belligerence.
      Do not mistake your goals as the only goals; your opinion as the only opinion; your confidence as correctness. Saying you know better is not the same as explaining you know better.

Re: No plan "weakens" my Test scripts?
by xdg (Monsignor) on Feb 08, 2005 at 13:41 UTC

    'no_plan' is good when you're willy-nilly writing tests and don't want to keep jumping to the top of your file to update the number of tests. (Alternative frameworks like Test::Class actually help keep count for you and it plays nicely with Test::More and friends).

    If you have a knowable number of test cases, you can always count the number and pass that to the plan (either on the "use" line or with the "plan" function. For example:

    use Test::More; open my($fh), "<datafile"; my $count = do { my $i; $i++ while <$fh>; $i }; seek ( $fh, 0, 0 ); plan tests => $count; load_database("datafile"); # or whatever while (<$fh>) { ok( in_database( $_ ) ); }

    -xdg

    Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

      my $count = do { my $i; $i++ while <$fh>; $i };
      my $count = do { () = <$fh>; $. };

      Makeshifts last the longest.

        I'm pretty sure that () = <$fh> loads the entire file contents into memory at once (on the stack), not throwing any of it away until the entire file has been read. So you might as well just use:

        my $count= ()= <$fh>;

        If you want to avoid memory use:

        my $count= do { 0 while <$fh>; $. };

        - tye        

Re: No plan "weakens" my Test scripts?
by Anonymous Monk on Feb 08, 2005 at 09:16 UTC
    As a contrived example, what if I want to test that every record in a file got loaded to a database? [...] I don't know how many records will be in the file, so I won't know how many tests will be executed before hand.
    If you don't know how many records are in the file, how do you know all the records got uploaded?
      In situations like that described, the idea is that your test script should *count* the number of records in the file (before running any tests), and then call plan(). Let's say your test script has 10 regular tests, plus it wants to run one test per line in somefile.txt. You'd use something like this:
      use Test::More; ## NB *not* no_plan open(FOO,"<somefile.txt") or die "Can't open test file: $!" my $a = 0; $a++ while (<FOO>); close(FOO); plan tests => 10 + $a;
        Or just:
        use Test::More; open my $foo, "<", "somefile.txt" or die; () = <$foo>; plan tests => 10 + $.;
        And if your tests aren't going to run on systems that don't have wc:
        {no warnings; use Test::More plan => 10 + `wc -l "somefile.txt"`}

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://428930]
Approved by davido
Front-paged by grinder
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others having an uproarious good time at the Monastery: (1)
As of 2024-04-18 23:46 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found