Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Inverting test conditions in Test::More ?

by LanX (Cardinal)
on Jul 24, 2020 at 08:51 UTC ( #11119738=perlquestion: print w/replies, xml ) Need Help??

LanX has asked for the wisdom of the Perl Monks concerning the following question:

Hi

What's the recommended way to invert a test? IOW I want it to fail?

E.g Test::More has is_deeply() but no is_not_deeply()

See Re: Test scalar refs (not values) in deeply nested structure (workaround) for examples which need to fail.

Do I really need to redirect the TAP output and parse it again?

Not looking for workarounds for is_deeply but for general test logic. ( I could just serialize a nested structure with Data::Dump and compare the strings)

Cheers Rolf
(addicted to the Perl Programming Language :)
Wikisyntax for the Monastery

  • Comment on Inverting test conditions in Test::More ?

Replies are listed 'Best First'.
Re: Inverting test conditions in Test::More ?
by hippo (Chancellor) on Jul 24, 2020 at 09:59 UTC

    There doesn't seem to be a simple, generic way to negate an arbitrary test in Test::More AFAICT. But, if you are happy enough to dip your toe in the Test2::Suite waters:

    use strict; use warnings; use Test2::V0; use Test2::Tools::Compare qw/is isnt/; my $w = { z => [ 0 .. 5 ] }; my $x = { z => [ 0 .. 5 ] }; my $y = { z => [ 1 .. 5 ] }; is ($w, $x, 'Match is good'); isnt ($x, $y, 'Not matching - also good'); done_testing;

    🦛

      Thanks, finally someone accepting my question! :)

      > There doesn't seem to be a simple, generic way to negate an arbitrary test in Test::More AFAICT.

      I just realized that all tests return a boolean result.

      So here a workaround:

      use strict; use warnings; use Test::More; sub fail_ok (&;@) { my ($code,$description) = @_; my $result; TODO: { local $TODO= "should be silent"; my $result = $code->(); } $result ? fail($description) : pass($description); } fail_ok { is(1,2,"") } "Bingo! Test failed"; done_testing;

      The TODO block ensures that there is no failed test in the final result.

      But I still need to understand how to make the output of the failed inner test silent ... °

      ... I probably need to redirect the STDERR output of Test::Builder or fiddle with TAP altogether

      -*- mode: compilation; default-directory: "d:/exp/" -*- Compilation started at Fri Jul 24 13:02:20 C:/Perl_524/bin\perl.exe -w d:/exp/pm_negate_test.pl not ok 1 - # TODO should be silent # Failed (TODO) test '' # at d:/exp/pm_negate_test.pl line 21. # got: '1' # expected: '2' ok 2 - Bingo! Test failed 1..2 Compilation finished at Fri Jul 24 13:02:21

      Cheers Rolf
      (addicted to the Perl Programming Language :)
      Wikisyntax for the Monastery

      °) Hmm... probably with subtests...

        Are you testing regular code, or testing other tests?

        If the former, why not just something like
        my $cond = 1 == 2; ok( $cond ); # fails ok( ! $cond ); # passes
        ... or whatever boolean condition.

        If the latter, Test::Builder::Tester seems designed to test other tests.

        To be fair, I could be misunderstanding - this is not a use case I've come across before.
        > But I still need to understand how to make the output of the failed inner test silent ... °

        FWIW, this will silence the output, tho using a new Test::Builder object or using Test::Builder::Tester might be the better approach.

        use strict; use warnings; use Test::More; my $Test = Test::Builder->new; sub fail_ok (&;@) { my ($code,$description) = @_; # --- silence output my $ignore; $Test->output(\$ignore); $Test->failure_output(\$ignore); $Test->todo_output(\$ignore); my $result; TODO: { local $TODO= "should be silent"; #my $result = subtest "Negation of $description", $code; my $result = $code->(); } # --- undo silence $Test->reset_outputs; # --- output result $result ? fail($description) : pass($description); } fail_ok { is(1,2,"") } "Bingo! Test failed"; fail_ok { is(1,3,"") } "Bingo! Test failed again"; done_testing;

        C:/Perl_524/bin\perl.exe -w d:/exp/pm_negate_test.pl ok 2 - Bingo! Test failed ok 4 - Bingo! Test failed again 1..4

        Cheers Rolf
        (addicted to the Perl Programming Language :)
        Wikisyntax for the Monastery

Re: Inverting test conditions in Test::More ?
by Corion (Pope) on Jul 24, 2020 at 09:05 UTC

    Why don't you test directly for where you know the difference is? You won't need is_deeply since you don't need to check all elements. You only need to check that there is at least one difference, and if you know your input data, you know where to find the difference.

      As I said

      > > Not looking for workarounds for is_deeply but for general test logic.

      Cheers Rolf
      (addicted to the Perl Programming Language :)
      Wikisyntax for the Monastery

        What part of "instead of checking whether there is any difference, check for a specific difference" is specific to is_deeply and not general test logic?

        Sounds like you're just trying to illicit specific responses. Better to just come out and say what you want to say rather than attempt to be coy about it.
Re: Inverting test conditions in Test::More ?
by perlfan (Priest) on Jul 24, 2020 at 09:15 UTC
    Looks like you could easily use Test::More::_deep_check to create is_not_deeply and submit a PR. Of course, either you're the first person ever to need this or there's something fundamentally wrong with whatever it is that you're doing. Is there a reason you must do "is_not_deeply" when a test to make sure "is_deeply" would suffice? I'm still not quite sure what it is you're testing.

    >I could just serialize a nested structure

    If you can ensure consistent/ordered serialization in the case of your structures being hash refs (or use Tie::IxHash somehow, I suppose. But this doesn't seem to be the right approach if you're reaching for deeply comparing two structures. Consider that you'd probably need to write unit tests to test that your serialization is coming out as expected.

      Supposing I just wanted to test deeply nested structures ...

      > Looks like you could easily use Test::More::_deep_check

      that's an internal function, Test::More provides explain() for serialization within the regular API

      see also eq_array() and eq_hash() but "they may be deprecated in future versions."

      > If you can ensure consistent/ordered serialization in the case of your structures being hash refs

      Please note that the order of hash-keys within the same Perl run is guarantied to be stable.

      Furthermore is explain() ordering hash-keys alphabetically AFAICT.

      ( Data::Dump too IIRC )

      Cheers Rolf
      (addicted to the Perl Programming Language :)
      Wikisyntax for the Monastery

      > Of course, either you're the first person ever to need this or there's something fundamentally wrong with whatever it is that you're doing

      Applying boolean logic to tests seems to be a straightforward idea.

      Just checking if I missed it to be documented / implemented.

      > submit a PR.

      A press release???

      Cheers Rolf
      (addicted to the Perl Programming Language :)
      Wikisyntax for the Monastery

        A press release???

        A pull request AKA a merge request. It's what all the cool kids are using instead of patches these days. :-)


        🦛

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://11119738]
Approved by marto
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others having an uproarious good time at the Monastery: (5)
As of 2020-09-19 03:46 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    If at first I don’t succeed, I …










    Results (114 votes). Check out past polls.

    Notices?