http://qs321.pair.com?node_id=587355

skazat has asked for the wisdom of the Perl Monks concerning the following question:

Heya,

Does anyone know of a swanky web-based way to run perl test files?

The situation I'm facing is this:

* I'd like to deploy the tests on a web-based application, making sure nothing fails, when I do an actual install for this app - installation mainly means, uploading some files via FTP, setting some config parts and changing the permissions of the main script. Pretty easy stuff.

* My development "community" isn't really that active in checking out dev. copies of the program, running the tests included and reporting back, so this usually falls on my back. I personally don't have a million different testing platforms, so it becomes difficult to mark something as, "stable", even if it passes tests on my few different environments. I also don't have an incentive to purchase or cobble together different environments, because of laziness and lack of monetary resources.

* Sometimes I don't have access to a CLI -0 el-cheapo web hosts sometimes don't give out ssh access, so I thought having a web-based way would save me some trouble, since the app is mostly a web-based app anyways. I was looking for something pretty polished, as it would be an incentive to bundle the test suite (a directory of .t files) with the app itself and have a friendly way for any user to run the tests out and report back.

Basically, hacking together this:

#!/usr/bin/perl -w use CGI qw(:fatalsToBrowser); use CGI qw(:standard); $|++; my $Test_Files = './t'; print header(); print "<h1>Starting...</h1>"; my $test_files = get_test_files(); foreach $file(@$test_files){ print "<h2>$file</h2>"; print "<hr><pre>"; # print `perl $Test_Files/$file`; print `prove -r -v $Test_Files/$file`; print "</pre>"; } print "<h1>Done!</h1>"; sub get_test_files { my @tests; if(opendir(TESTDIR, $Test_Files)){ my $tf; while(defined($tf = readdir TESTDIR) ) { next if $tf =~ /^\.\.?$/; $tf =~ s(^.*/)(); if($tf =~ m{\.t$}){ push(@tests, $tf); } } closedir(TESTDIR) or warn "couldn't close: " . $Test_Files; } return \@tests; }

Does pretty much what I want, albeit in a not so very pretty way, but that's easily polished up. Some problems I can see happening could possibly be: browser timeouts from a script that takes a long time running all the tests, a screen of HTML that seems to never end, misc. problems that aren't really reported in the browser, environment differences between running a script via the CLI and your web browser, etc.

Has anyone thought of a more interesting solution?

Again, I just want to *run* tests in my web-browser, I don't want to run tests *as* a web browser.

 

-justin simoni
skazat me

Replies are listed 'Best First'.
Re: Running .t Tests in a web-based environment
by chromatic (Archbishop) on Dec 02, 2006 at 08:46 UTC

    I had a different solution in my Perl Hacks talk this summer; I'll write it up and post it somewhere soon.

    Meanwhile, I suggest making the web process generate plain TAP and running that through Test::Harness yourself elsewhere. That ought to avoid most of the tricky problems there. Perhaps you can have an extra directory of applications, properly password-protected of course, that run the tests for you.

    (Be sure to redirect STDERR and STDOUT to the same place before you load any test modules, or else you likely won't get diagnostic output.)

Re: Running .t Tests in a web-based environment
by SheridanCat (Pilgrim) on Dec 02, 2006 at 05:11 UTC
    Take a look at using Test::Harness for this type of testing. Here's a bit of code I use to run a bunch of tests and get a nice report (it's inspired by chromatic's Perl Testing ISBN 0596100922): I'm not sure how I'd handle the long running tests. Maybe output the results to a file you can come back and pick up later. The key is getting the browser to disconnect and keep the test running.
Re: Running .t Tests in a web-based environment
by fenLisesi (Priest) on Dec 02, 2006 at 14:10 UTC
Re: Running .t Tests in a web-based environment
by Rhandom (Curate) on Dec 03, 2006 at 02:44 UTC
    Every time I go to a new company I write a harness. And then a CGI that can run the tests from the web. I didn't know about /usr/bin/prove - now I don't need to write that anymore. But then I also use something similar to the following code. You will need CGI::Ex installed - but I think the results will be something you like.

    Oh - and a note - you will want to change BASE_DIR to the base directory where your tests are stored. It also then depends upon you putting your tests into subdirectories as in:
    /my/base_dir/t/ /my/base_dir/t/BaseSystems/ /my/base_dir/t/BaseSystems/0_foo.t /my/base_dir/t/BaseSystems/1_bar.t /my/base_dir/t/MainSystem/0_test_something.t


    Enjoy!

    #!/usr/bin/perl =head1 NAME cgi_prove - Web based interface to /usr/bin/prove =cut use base qw(CGI::Ex::App); use strict; use warnings; use vars qw($BASE_DIR $LIB_DIR $PROVE_PROG); use CGI::Ex::Dump qw(debug); BEGIN { require config; $BASE_DIR = $config::config{'rootdir_server'} .'/t'; $LIB_DIR = $config::config{'rootdir_server'} .'/lib'; $PROVE_PROG = '/usr/bin/prove'; } ###----------------------------------------------------------------### __PACKAGE__->navigate; sub main_file_print { return \ qq { <h1>Run t/tests <span style=font-size:smaller>(<a href=[% scri +pt_name %]?test=all>Run All</a>)</span></h1> [% rows = []; FOREACH dir IN files.keys.sort ; rows.push("<b><a href=\$script_name?test=\${dir.uri}>\${d +ir.html}</a></b><br>"); FOREACH file IN files.\$dir ; rows.push("&nbsp;&nbsp;&nbsp;&nbsp;<a href=\$script_nam +e?test=\${dir.uri}/\${file.uri}>\${file.html}</a><br>"); END; END; cols = 4; n_per_col = rows.size / cols; IF n_per_col - n_per_col.int; n_per_col = n_per_col.int + +1; END; FOREACH j = [0 .. cols - 1] %] <div style="float:left; width:24%"> [% min = j * n_per_col; max = min + n_per_col - 1; IF max > rows.max; max = rows.max; END; FOREACH i IN [min .. max] rows.\$i; END %] </div> [% END %] }; } sub main_hash_swap { my $self = shift; my $dir = $BASE_DIR; require File::Find; my $files = {}; File::Find::find(sub { return if -d; return if $File::Find::name !~ m|^\Q$dir\E/(.+)/(.+\.t)$|; $files->{$1}->{$2} = 1; }, $dir); $files->{$_} = [sort keys %{$files->{$_}}] foreach keys %$files; return {files => $files}; } sub main_info_complete { my $self = shift; my $file = $self->form->{'test'} || return 0; return 0 if $file =~ /\.\./; return 0 if $file !~ /^([\w\.\/\-]+)$/; $file = $1; $file =~ s|^/+||; my $dir = $BASE_DIR; return 0 if ! -e "$dir/$file" && lc $file ne 'all'; $self->stash->{'test'} = $self->form->{'test'} = $file; $self->append_path('_run'); return 1; } sub _run_file_print { return \ qq { <h1>Test - [% test %]</h1><br> }; } sub _run_info_complete { 0 } sub _run_post_print { my $self = shift; my $file = $self->stash->{'test'} || die "Missing test"; my $exe = $PROVE_PROG; $file = '' if lc($file) eq 'all'; open(my $fh, "$exe -v -I$LIB_DIR -r $BASE_DIR/$file 2>&1 |") || do + { debug "Couldn't exec", $exe, $!; die "Couldn't exec: $!" }; my $r; if ($self->cgix->is_mod_perl_2) { $r = $self->cgix->apache_request; } else { $| = 1; } print "<span style=\"font-family:monospace;color:green\">\n"; while (defined(my $line = <$fh>)) { $line =~ s/\s+$//; if ($line =~ /^\s*not/) { $line = "<span style=color:red>$line</span>"; } elsif ($line !~ /^\s*ok/) { $line = "<span style=color:blue>$line</span>"; } print $line."<br>\n"; $r->rflush if $r; } print "</span>\n"; }


    my @a=qw(random brilliant braindead); print $a[rand(@a)];
Re: Running .t Tests in a web-based environment
by adrianh (Chancellor) on Dec 03, 2006 at 10:30 UTC

    You might want to take a look at Test::TAP::Model and friends to present the test result. My local Module::Build subclass has:

    sub _do_action_with_tap_model { my ( $self, $action ) = @_; require Test::TAP::Model::Visual; require Test::TAP::HTMLMatrix; require File::Path; File::Path::mkpath( 'html/t' ); my $model = Test::TAP::Model::Visual->new or croak "could not make Test::TAP::Model::Visual"; { require Test::Harness; no warnings 'redefine'; my $original = \&Test::Harness::runtests; local *Test::Harness::runtests = sub { local *Test::Harness::runtests = $original; $model->run_tests( @_ ) }; print "starting $action with html test results\n"; $self->$action( @_ ); print "finished $action with html test results\n"; } my $html_matrix = Test::TAP::HTMLMatrix->new( $model ) or croak "could not make Test::TAP::HTMLMatrix"; open( my $fh, '>', 'html/t/index.html' ) or croak "open failed ($! +)"; print $fh $html_matrix->html or croak "print failed ($!)"; close $fh or croak "close failed ($!)"; print "test results written to html/t/index.html\n"; } sub ACTION_testhtml { my $self = shift; $self->_do_action_with_tap_model( 'ACTION_test' ); } sub ACTION_smoke { my $self = shift; $self->_do_action_with_tap_model( 'ACTION_testcover' ); }

    While allows me to do things like ./Build testhtml to get pretty HTML results in html/t.