Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

C test suite management with perl

by QuillMeantTen (Friar)
on Sep 09, 2015 at 16:46 UTC ( [id://1141444]=CUFP: print w/replies, xml ) Need Help??

Greetings fellow monks,
for my studies at the university I recently endeavored with a friend to prepare the best infrastructure for the long term projects we are to be given later this year.
Amongst other things we set up a continuous integration environment, an xmpp server for team communications and that kind of things.
Since we both are big fans of test driven development and I fell in love with the TAP protocol as I started writing my own modules I decided to give this library a shot. After some manipulations I got it to work and started writing on scripts that would allow us to easily build and centralize test logs.

Thing is in previous projects we did that kind of thing in big and unwieldy bash scripts. I decided to do it in perl but hit quite the roadblock : my friend does not know perl and has no intention to learn it in the foreseeable future.

I prepared the following solution :

One makefile that calls the build.pl script (with whatever parameter are needed) and a run_test.pl script that goes through all the files in the tests directory, run all tests, log results, if some tests fail it creates two files, one for all failed test for the file tested and another with all the tests for the file tested to give it some context.

here is build.pl

#!/usr/bin/perl use strict; use warnings; use File::Path; use Cwd; if ($ARGV[0]=~ 'all'){ mkdir 'logs'; chdir 'logs'; my $logwd = getcwd(); chdir '..'; my $basewd = getcwd(); chdir 'tests'; my @files = glob "*"; foreach my $file(@files){ my $outwd = $logwd.'/'.$file; if(-d $file){ mkdir "$logwd/$file"; chdir $file; if(`../run_tests.pl $outwd` =~ 0){ rmtree($outwd); } } } chdir $basewd; `make unchecked`; } elsif($ARGV[0]=~/clean/){ my @ofiles = glob "*.o"; foreach my $file(@ofiles){ print "cleaning $file\n"; unlink $file; } }

here is run_tests.pl :

#!/usr/bin/perl use strict; use warnings; `make tests`; my $logwd = $ARGV[0]; my $pid = open(my $kid_to_read,"-|"); if($pid == 0){ print `./tests`; } else{ open(my $failed_tests,'>',"$logwd/test_fail.log"); open(my $test_output,'>',"$logwd/test_out.log"); my $fail = 0; while(<$kid_to_read>){ my $line = $_; if($line =~ /\Anot ok/){ print $failed_tests $line; $fail++; } print $test_output $line; } close $test_output; close $failed_tests; `make clean`; print $fail; }

and the main project makefile :

all: ./build.pl all make clean .PHONY: clean clean: ./build.pl clean unchecked:

The unchecked target is used to build the project without running the test scripts.
It is important to take into account that inside the tests folder each subfolder contains its own makefile to build the test executable and then clean after tests have been run.

As always I post here hungry for ways to make that code better, more efficient or solve overlooked issues or design problems.
Also I hope that someone might have a use for it ^^
Kind Regards.

Replies are listed 'Best First'.
Re: C test suite management with perl
by jeffa (Bishop) on Sep 09, 2015 at 17:39 UTC

    Why not just use prove? If you just want to run your tests, and you don' t care about the order they are run in (and you should not to take advantage of forked parallelization) then prove will handle pretty much everything your run_tests.pl script does.

    I put all my tests into a directory named t/ inside the project directory and run

    prove -rlj6
    from the project directory or
    prove -rlj6 --formatter=TAP::Formatter::JUnit --timer t/| tee junit.xm +l
    if i want to feed the results to a CI server like Jenkins, so it can build pretty graphs for me.

    If you want to build an executable then perhaps you can leverage Makefile.pl instead of rolling your own wheel from complete scratch. I recommend Module::Starter to start any and all projects. Once you have a proper Makefile.pl, then you can use the shell to replace build.pl

    #!/bin/sh perl Makefile.PL make make test # do something with newly created files make veryclean
    If you need more control, then i suggest starting with a foundation like App::Prove.

    Hope this helps, there are many branches to hit when falling down this tree. :)

    jeffa

    L-LL-L--L-LL-L--L-LL-L--
    -R--R-RR-R--R-RR-R--R-RR
    B--B--B--B--B--B--B--B--
    H---H---H---H---H---H---
    (the triplet paradiddle with high-hat)
    

      I thought about that but I could not find a way to do it with C test files, at first I tried using Inline to be able to use prove easily but as more and more difficulties befell me I gave up in favor of the c library I linked
      I would love to know how to use module starter to make non perl projects though, would you be so kind as to expand on that?

        From the link you provided:

        C TAP Harness can run Perl test scripts directly and interpret the results correctly, and similarly the Perl Test::Harness module and prove command can run TAP tests written in other languages using, for example, the TAP library that comes with C TAP Harness. You can, if you wish, use the library that comes with C TAP Harness but use prove instead of runtests for running the test suite.

        jeffa

        L-LL-L--L-LL-L--L-LL-L--
        -R--R-RR-R--R-RR-R--R-RR
        B--B--B--B--B--B--B--B--
        H---H---H---H---H---H---
        (the triplet paradiddle with high-hat)
        

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: CUFP [id://1141444]
Approved by herveus
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others lurking in the Monastery: (9)
As of 2024-04-18 14:13 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found