run tests only changed files

mandog has asked for the wisdom of the Perl Monks concerning the following question:

My test suite takes about 6 minutes to run. Most of the time, I'm testing things that haven't changed.

Is make the tool to run my tests faster?

Does a script to regenerate my makefile already exist?

Translating My::Foo:Thing to 'lib/Foo/Thing.pm' doesn't look very hard. The Test::WWW::Mechanize tests aren't as obvious but eyebballing the output from Devel::NYTProf or Devel::Cover would be a start

Obviously make clean would be a good idea before pushing to production

Comment on run tests only changed files Select or Download Code

Replies are listed 'Best First'.
Re: run tests only changed files by Your Mother (Archbishop) on Aug 07, 2015 at 14:13 UTC
`make` doesn't do anything about test speed. The App::Prove tools can help you though. Make sure you have a relatively new version of prove installed and read up on all its options with a focus on these– `-j, --jobs N Run N test jobs in parallel (try 9.) --state=opts Control prove's persistent state. --rc=rcfile Process options from rcfile --rules Rules for parallel vs sequential proces +sing.` [download]	[reply] [d/l] [select]
Re: run tests only changed files by Monk::Thomas (Friar) on Aug 07, 2015 at 13:59 UTC
My test suite takes about 6 minutes to run. Most of the time, I'm testing things that haven't changed. Are you sure nothing has changed? 'Action at a distance' comes to mind. I run tests in order to prove that nothing has changed. Maybe you can optimize the tests and make them run faster? If that is not an option then you could run a specific test file, e.g.: `perl -I lib/ t/10_lulu.t` [download] ...and as soon as it yields the expected result run the whole testsuite.	[reply] [d/l]
Re: run tests only changed files by hippo (Bishop) on Aug 07, 2015 at 14:01 UTC
Most of the time, I'm testing things that haven't changed. Most of the time you are testing things which you think haven't changed. There's a difference. If you really only want to test the bits which you think have changed, then just run those test files. How many files are there in your test suite? Just run the one which tests what you think has changed and only run the full test suite when the individual script passes. Or have the full test suite run on-demand but in the background so you can get on with other things. There are so many ways to approach this without going to the complexity of dependency-driven tests.	[reply]
Re: run tests only changed files by CountZero (Bishop) on Aug 07, 2015 at 16:20 UTC
You don't even have to change things yourself for matters to go wrong. Perhaps some sys-admin has "updated" something somewhere or changed some network configuration and all of a sudden your scripts stop working. You really want to find out before you have to run that business and time critical script. CountZero A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James My blog: Imperial Deltronics	[reply]
Re: run tests only changed files by anonymized user 468275 (Curate) on Aug 07, 2015 at 14:33 UTC
Yes, makefiles can be written to do just that. The secret is to define rules for separately named subtargets. Update: re how much to test, testing only changed file impact is fine for unit testing of changes. But when you are regression testing them, sundial's comments below will of course apply. But even if that were true, you still don't need to rebuild targets whose dependencies haven't changed. The exception is of course where you updated the makefile, then you have to clean and rebuild everything! One world, one people	[reply]
A reply falls below the community's threshold of quality. You may see it by logging in.

Back to Seekers of Perl Wisdom