Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Quiet test runner

by adrianh (Chancellor)
on Jul 30, 2005 at 14:39 UTC ( [id://479609]=sourcecode: print w/replies, xml ) Need Help??
Category: testing
Author/Contact Info Adrian Howard
Description:

Quick and dirty test runner for perl that only reports test failures and unexpectedly succeeding todo tests.

Update: typo fixed, thanks tlm.

Update: fixed is_expected_test_result, thanks tomhukins

#!/usr/bin/perl

use strict;
use warnings;
use Test::Harness::Straps;
use IPC::Run3;

my $strap = Test::Harness::Straps->new();

show_failures( $_ ) for @ARGV;

sub show_failures {
    my $file = shift;
    my $result = test_result( $file );
    show_details( $result ) unless is_expected( $result );
}

sub test_result {
    my $file = shift;
    return { 
        file => $file,
        $strap->analyze( $file, collect_stderr_and_stdout( $file ) )
    };
}

sub is_expected {
    my $result = shift;
    return $result->{ passing } && $result->{ bonus } == 0;
};

sub show_details {
    my $result = shift;
    print "$result->{file}\n", 
        map { "* $_->{name}\n" . diagnostics( $_ ) } 
            grep { ! is_expected_test_result( $_ ) } @{ $result->{ det
+ails } };
}

sub is_expected_test_result {
    my $test_result = shift;
    return $test_result->{ok} &&
        ! is_unexpected_todo_success( $test_result ) &&
        ! diagnostics( $_ );
}

sub diagnostics {
    my $test_result = shift;
    return $test_result->{ diagnostics } if $test_result->{ diagnostic
+s };
    return "    unexpected success from TODO test\n"
        if is_unexpected_todo_success( $test_result );
    return '';
}

sub is_unexpected_todo_success {
    my $test_result = shift;
    return $test_result->{type} eq "todo" && $test_result->{actual_ok}
}

sub collect_stderr_and_stdout {
    my $file = shift;
    my $command = [ $strap->_command, $strap->_switches( $file ) || ()
+, $file ];
    my $output = [];
    run3( $command, \undef, $output, $output ) || die "run3: $!\n";
    return $output;
}
Replies are listed 'Best First'.
Re: Quiet test runner
by tomhukins (Curate) on Sep 21, 2005 at 15:22 UTC

    I find this really useful, thanks. I'm using it as a consequence of the sitation I described in Reporting test failures within cron.

    I noticed it doesn't work quite right if I run an incorrect number of tests. For example:

    use Test::More tests => 3; ok 1, 'Happy'; ok 1, 'Cheerful';

    We expect three tests but only run two, which both succeed. Fortunately, is_expected deals with this, but unfortunately show_details doesn't report the failure. The altered is_expected_test_result below fixes this reporting problem:

    sub is_expected_test_result { my $test_result = shift; return $test_result->{ok} && ! is_unexpected_todo_success( $test_result ) && ! diagnostics( $_ ); }

    I discovered this when running some Test::WWW::Mechanize tests against a server that stops responding, causing later tests not to run.

      Well spotted. Thanks.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: sourcecode [id://479609]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others contemplating the Monastery: (3)
As of 2024-04-24 01:27 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found