http://qs321.pair.com?node_id=11100090


in reply to Bailing out from Test::More after 'some' failures

Take a look at Test2::Plugin::BailOnFail. You can probably copy that and change it so that it counts failures and fails after N instead of failing after the first one. Test::Builder is now a compatibility layer around Test2 these days so this will count failures Test::Builder will miss. Also this plugin does not have the problem of loosing diagnostics (most of the time) that the Test::Builder override does that requires the override to wait for the next assertion before exiting. Finally this way of doing it requires no hacks or monkey-patching, instead it uses things put in place to intentionally make what you want possible.
  • Comment on Re: Bailing out from Test::More after 'some' failures

Replies are listed 'Best First'.
Re^2: Bailing out from Test::More after 'some' failures
by choroba (Cardinal) on May 16, 2019 at 14:44 UTC
    I'm intrigued by the "most of the time" parenthesis. Could you elaborate?
    map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]
[reply]
[d/l]
      Most of the time means that any tool written with Test2 in mind, or directly on Test2 will work as expected, this should include most methods on Test::Builder itself. However cpan tools written against old Test::Builder that do not use Test2's 'Context' system may still call $tb->ok and $tb->diag separately, in which case the diag will still be lost.

      In Test2 you would do this: 
          sub mytool {
      my $ctx = context();
      $ctx->ok(...);
      $ctx->diag(...);
      $ctx->release; # This is where the bail-on-fail is triggered
    }

      [download]
      The TB version: 
          sub mytool {
      $tb->ok(...); # BailOnFail is triggered here
      $tb->diag(...); # Lost
    }

      [download]
      $tb methods do acquire context's, but each of those methods has to acquire it's own. That said the diagnostics TB itself generates in a failed-ok (like line number) will make the output, it is just extra diags are lost.

      That is unfortunately a fundamental, and not-fixable limitation of Test::Builder that Test2 avoided with the context system. Test2 was specifically designed to overcome this and many other TB limitations.
[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom