Universal test flag

ait has asked for the wisdom of the Perl Monks concerning the following question:

Hi. I'm looking for a universal way to declare a test flag to object oriented code regardless if it's Moose or traditional Perl OO, or whatever, and although rare, even if the blessed reference is an array. I'm pretty sure I'm reinventing the wheel here so the code below is merely to illustrate what is wanted.

Anyway, the idea is that from a test script one could can call this method and it would set a test flag that can later be used in the actual code to condition things or in cases the code needs to know it's being tested. So, the test script would call _test_mode to set the attribute and then the actual code will use _test_mode to condition things (e.g. do foo unless($self->_test_mode)). This could perhaps be added as a simple sub in in modules that need to be test-aware:

sub _test_mode {
  my $self = shift;
  use Scalar::Util 'reftype';
  my $tkey = '_'.__PACKAGE__.'_TEST_MODE';
  if(reftype($self) eq 'HASH'){
    if(defined $self->{$tkey}){
      return 1;
    }
    else{
      $self->{$tkey} = 1;
    }
  }
  elsif(reftype($self) eq 'ARRAY'){
    if($self->[-1] eq $tkey){
      return 1;
    }
    else{
      push @{$self},$tkey;
    }
  }
}
[download]

There are probably dozens of ways to do this, and more that one published on the CPAN, so I ask for your Wisdom not so much on the code above but rather on recommendations on techniques to accomplish the objective of setting a universal flag in the code so it's aware of being tested, regardless if we are testing a module built with Moose, or an extension to some DBIx::Class:ResultSet.

Comment on Universal test flag Download Code

Replies are listed 'Best First'.
Re: Universal test flag by bulk88 (Priest) on Jul 17, 2012 at 05:35 UTC
I suggest using subroutine constants that are set in a BEGIN block before the module is loaded with use. This way the test mode/debugging code has no performance or memory hit at runtime due to constant folding. Make sure to check with B::Deparse or B::Concise that test mode code was fully removed from the opcode tree, it is very easy to break constant folding. I have a problem with seeing 2-8 parameter debug subroutine calls every 3-6 lines in a CPAN module to enable console printing debugging (as if Perl doesn't have a debugger with a watch window).	[reply]
Re^2: Universal test flag by ait (Hermit) on Jul 17, 2012 at 13:17 UTC
Thanks for your comment. Yes, I understand your concern and I second your opinion that print statements for debugging are a bad idea, one of the reasons I looked into using callbacks that could allow the test software to inspect intermediate results within a sub. There are some situations (e.g. "large amount of state data shared among the different steps of a function" see 880089) in which intermediate values inside a sub need to be evaluated, but instead of printing these ase debug statements I have been using a "test point" analogy that I describe here 882331. This allows a your test to diagnose the problem better before firing up the debugger to fix it. Of course many of this situations can be avoided with better design, but in the real world time and budget constraints don't allow for adequate refactoring to fix these design problems. The test point idea was precisely for eliminating all these print-debug statements and move these intermediate values to be evaluated in the test suite instead of reading and making sense of all the verbose debug statements.	[reply]
Re^3: Universal test flag by bulk88 (Priest) on Jul 17, 2012 at 15:19 UTC
I dont see Using Test-Point Callbacks for Automated Diagnostics in Testing as being any more efficient than a print statement since multiple arguments of lexicals are still evaluated and copied if the branch is taken and those instructions will not be removed, although they wont execute, with debugging mode off. The only advantage I see of Using Test-Point Callbacks for Automated Diagnostics in Testing is that errors can be discovered automatically with code, rather than the programmer looking for a needle in a haystack wrong value in a mile of debug channel.	[reply]
Re^4: Universal test flag by ait (Hermit) on Jul 17, 2012 at 16:06 UTC
Re: Universal test flag by tobyink (Canon) on Jul 17, 2012 at 08:21 UTC
`package Foo::Bar; use constant DEBUG => $ENV{DEBUG_FOO_BAR}; # ... warn "I'm being tested!" if DEBUG;` [download] Then if you need a particular class to be debugged, set the appropriate environment variable to "1" before the class is loaded. `perl -E'sub Monkey::do{say$_,for@_,do{($monkey=[caller(0)]->[3])=~s{::}{ }and$monkey}}"Monkey say"->Monkey::do'`	[reply] [d/l]
Re^2: Universal test flag by ait (Hermit) on Jul 17, 2012 at 13:23 UTC
Thanks! The ENV approach is really simple and clear can be set from the test script before loading the class.	[reply]
Re: Universal test flag by zwon (Abbot) on Jul 17, 2012 at 13:24 UTC
This approach has a problem -- you won't be testing production code with this flag set e.g.: `if ($self->_test_mode) { # test_code } else { # production code, not tested }` [download] If code needs to know that it is being tested, I suspect something wrong with the interface. Also, may be you should have a look onto mocking modules like Test::MockObject.	[reply] [d/l]
Re^2: Universal test flag by ait (Hermit) on Jul 17, 2012 at 14:24 UTC
If code needs to know that it is being tested, I suspect something wrong with the interface. Probably so, and in theory everything sound nice, but in real-world every-day coding, with a team of coders, and with time and budget constraints, this is by far the common case. You need to be able to test things while skipping others that don't concern that particular aspect that you are working on or testing. Example: you are testing a class that invokes an external service to send an SMS, and the SMS gateway has no test mode, so it costs you money. Your class does a hundred things besides sending the SMS and has a lot of business logic and you only want to skip the SMS sending part to test this logic. Maybe the SMS sending logic was encapsulated in a class (in which case could be mocked) or maybe it's part of the same class on a sub and can't be mocked. Like this one, there are a many day-to-day scenarios where you want to skip over some part of the code just o test the overall logic. Refactoring to make code perfectly testable is not always an option, in fact, it rarely is, at least in large projects. The code is far from perfect and you have to make the best of it and try to test it as best you can within the time and budget constraints that a particular project allows.	[reply]
Re^3: Universal test flag by chromatic (Archbishop) on Jul 17, 2012 at 17:02 UTC
Refactoring to make code perfectly testable is not always an option... ... but writing more code is an option? If you have to write extra code in the code you're testing that only gets executed in test mode, I think your confidence in the test results goes down.	[reply]
Re^3: Universal test flag by zwon (Abbot) on Jul 17, 2012 at 18:25 UTC
Refactoring to make code perfectly testable is not always an option, in fact, it rarely is, at least in large projects The purpose of refactoring is to make code structure optimal, testability is just a side effect. And it is especially important for large projects, without maintaining code structure you will soon finish with spaghetti which will take years to unravel. Particularly, if you have classes which do hundred things and send SMSes, it does sound alarming.	[reply]
Re^3: Universal test flag by ait (Hermit) on Jul 18, 2012 at 13:42 UTC
chromatic and zwon: Obviously your world seems more perfect than ours, but the hard truth in the commercial world is that there is always a tradeoff between quality and the amount of money and time available. Most of the projects today are constrained in both, and you must quickly adapt constantly changing business conditions, so refactoring always falls in second place, after you get the (ever changing) functionality pinned down. Testing on the other hand is paramount to make sure each piece does what it should and all the pieces fit together when several people are working on the same project. Of course, code must be good enough to be tested but it surely doesn't have to be perfect. Sure, software is perfectible if you have enough time and resources, but the cold hard truth is that in most situations customers are not willing to pay the extra money for perfect code and their budget only allows for "good enough". IMHO, the truly successful business projects are able to deliver in time and money with good enough code to get business flowing and creating the cash flow necessary to eventually perfect the code.	[reply]
Re^4: Universal test flag by chromatic (Archbishop) on Jul 19, 2012 at 17:44 UTC
Re^5: Universal test flag by ait (Hermit) on Jul 19, 2012 at 18:46 UTC
Some notes below your chosen depth have not been shown here
Re^4: Universal test flag by zwon (Abbot) on Jul 18, 2012 at 17:08 UTC
Re^5: Universal test flag by ait (Hermit) on Jul 19, 2012 at 12:07 UTC
Some notes below your chosen depth have not been shown here


The stupid question is the question not asked
	PerlMonks