Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Quieting Test::More

by stevieb (Canon)
on Jul 03, 2017 at 23:26 UTC ( #1194120=perlquestion: print w/replies, xml ) Need Help??

stevieb has asked for the wisdom of the Perl Monks concerning the following question:

Test::More is verbose in its output as it should be. That said, when I have say a loop of tests that produces ~100 results, is there a way to flag it to say "don't say anything, except for my debug print statements"?

Being able to do so would allow me to troubleshoot specific areas of my test files without having to continuously manually edit a test file and move around nonsense like:

done_testing(); exit;

...lines.

Using something like perl t/100-test.t > /dev/null works well to display only the failed test results to STDERR, but that buries print statements.

What do you do to either stop your tests from running upon a failed one, or allow your print statements to be printed to STDOUT and bypass everything else?

Replies are listed 'Best First'.
Re: Quieting Test::More
by kcott (Bishop) on Jul 04, 2017 at 08:03 UTC

    G'day stevieb,

    "... is there a way to flag it to say "don't say anything, except for my debug print statements"?"

    You can do this with a combination of diag statements, SKIP: blocks, and a DEBUG flag. Here's a very simple, and highly contrived, example (pm_1194120.t):

    use Test::More tests => 2; SKIP: { skip 'Debugging', 0 unless $ENV{PM_1194120_DEBUG}; diag 'DEBUG MODE!'; diag 'Debug statement #1 (1 == 1): ', 1 == 1 ? 'TRUE' : 'FALSE'; diag 'Debug statement #2 (1 == 0): ', 1 == 0 ? 'TRUE' : 'FALSE'; } SKIP: { skip 'Debugging', 2 if $ENV{PM_1194120_DEBUG}; is(1, 1, 'Test: 1 == 1'); isnt(1, 0, 'Test: 1 != 0'); }

    Without the debug flag being set, here's verbose, non-verbose, and no STDOUT output (timing information elided):

    $ export PM_1194120_DEBUG=0 $ prove -v pm_1194120.t pm_1194120.t .. 1..2 ok 1 - Test: 1 == 1 ok 2 - Test: 1 != 0 ok All tests successful. Files=1, Tests=2, ... Result: PASS $ prove pm_1194120.t pm_1194120.t .. ok All tests successful. Files=1, Tests=2, ... Result: PASS $ prove pm_1194120.t > /dev/null $

    Here's the same commands, with the debug flag set:

    $ export PM_1194120_DEBUG=1 $ prove -v pm_1194120.t pm_1194120.t .. 1..2 # DEBUG MODE! # Debug statement #1 (1 == 1): TRUE # Debug statement #2 (1 == 0): FALSE ok 1 # skip Debugging ok 2 # skip Debugging ok All tests successful. Files=1, Tests=2, ... Result: PASS $ prove pm_1194120.t pm_1194120.t .. # DEBUG MODE! # Debug statement #1 (1 == 1): TRUE # Debug statement #2 (1 == 0): FALSE pm_1194120.t .. ok All tests successful. Files=1, Tests=2, ... Result: PASS $ prove pm_1194120.t > /dev/null # DEBUG MODE! # Debug statement #1 (1 == 1): TRUE # Debug statement #2 (1 == 0): FALSE $

    While that last one does exactly what you ask (i.e. nothing but debug statement output); consider whether you really want to throw away STDOUT (it's only a few lines and could provide useful feedback).

    Obviously, there's a huge number of variations on that theme. You could have more than one debug flag and they certainly don't need to be environmental variables. I've separated the diag statements from the tests, but there could be a mixture; and, not everything needs to be in a SKIP: block.

    See also: prove (for a variety of other options, '-b' is one I usually need); and Test::More (in particular, the Diagnostics, Conditional tests, and Test control sections).

    — Ken

Re: Quieting Test::More
by 1nickt (Abbot) on Jul 04, 2017 at 01:13 UTC

    Like the Anonymous One said, (and wasn't this an answer to an earlier question you had about Test::More?): Don't run your tests with perl foo.t. Test::More is meant to be used inside a testing harness such as is provided by prove. use prove.

    $ prove -lr t/050-types.t t/050-types.t .. ok All tests successful. Files=1, Tests=22, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.07 cusr + 0.00 csys = 0.09 CPU) Result: PASS
    Also, use note and diag and explain in tests, then the -v option to prove can be used to control your "debug" output as well as test output verbosity.


    The way forward always starts with a minimal test.
Re: Quieting Test::More
by haukex (Bishop) on Jul 04, 2017 at 09:35 UTC

    Is this the kind of thing you're looking for? See also reset_outputs in Test::Builder.

    #!/usr/bin/env perl use warnings; use strict; use Test::More tests=>200; use File::Spec; Test::More->builder->output(File::Spec->devnull); is 1,1 for 1..199; # lots of output is 1,2, 'test failure'; print "print\n"; warn "warn\n"; diag "diag"; note "note"; __END__ 1..200 # Failed test 'test failure' # at tst.pl line 11. # got: '1' # expected: '2' print warn # diag # Looks like you failed 1 test of 200.
Re: Quieting Test::More
by Anonymous Monk on Jul 03, 2017 at 23:45 UTC

      Although not completely, question was a bit rhetorical in hopes for explanations with examples...

Re: Quieting Test::More (purpose, diag)
by Anonymous Monk on Jul 04, 2017 at 00:02 UTC

    IMHO

    I think the real problem is using Test::More as a debugger

    I get how easy it is to start using testing modules for everything

    I get wanting document every debugging session , every development milestones,

    but doing it in one file is bad habit, as is doing it using testing modules , the modularity of your API suffers if there is no separation between testing and development

    the solution is smaller and smaller teeny tiny test files, like bug specific rt-666.t bug-frobnitz.t, even if 80%-99% of each test file are identical save for the input/output

    but if you're gonna debug learn the API :) https://metacpan.org/pod/Test::More#Diagnostics

    Update: Link fixed by GrandFather

      I don't think you are understanding completely what I'm asking/referencing here.

      I do not do many things within a single file. In fact, I have written accepted patches to ExtUtils::MakeMaker documentation that ensures that sub tests and sub directories are understood within the testing framework, to ensure compartmentalized testing is available, and documented.

      I write test files that are specific to a small piece of my software, eg t/05-sub_one.t, t/10-sub_two.t. It doesn't matter if the test file has two or 400 tests in it. I want to know how to shut things up so I can see my print statements only.

      You can verify if we're on the same page here or not by looking here.

      I feel that my APIs flourish by testing them the way I do. I am definitely open to comments in that regard though, if one looks at one of my APIs, then the tests, then responds about them directly.

      404 on the link... this message will (hopefully) self-destruct. I would have /msgd, but, anonymonk and all.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1194120]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (3)
As of 2020-09-19 03:58 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    If at first I donít succeed, I Ö










    Results (114 votes). Check out past polls.

    Notices?