Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

The wonders of Test::More

by syphilis (Archbishop)
on May 21, 2022 at 13:29 UTC ( #11144043=perlquestion: print w/replies, xml ) Need Help??

syphilis has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

Have a laugh at this piece of brilliance:
C:\>perl -MMath::MPFR -MTest::More -e "cmp_ok(Math::MPFR->new(2), '!=' +, Math::MPFR->new(2), 'test for blow up'); done_testing();" not ok 1 - test for blow up # Failed test 'test for blow up' # at -e line 1. Operation "eq": no method found, left argument in overloaded package Math::MPFR, right argument in overloaded package Math::MPFR at C:/perl-5.3 +4.0/site/lib/Test/Builder.pm line 1006. A context appears to have been destroyed without first calling release +(). Based on $@ it does not look like an exception was thrown (this is not + always a reliable test) This is a problem because the global error variables ($!, $@, and $?) +will not be restored. In addition some release callbacks will not work prop +erly from inside a DESTROY method. Here are the context creation details, just in case a tool forgot to c +all release(): File: -e Line: 1 Tool: Test::More::cmp_ok Here is a trace to the code that caused the context to be destroyed, t +his could be an exit(), a goto, or simply the end of a scope: Context destroyed at C:/perl-5.34.0/site/lib/Test/Builder.pm line 1006 +. eval {...} called at C:/perl-5.34.0/site/lib/Test/Builder.pm l +ine 1006 Test::Builder::cmp_ok(Test::Builder=HASH(0x24eaa60), Math::MPF +R=SCALAR(0x86b998), "!=", Math::MPFR=SCALAR(0x33cf88), "test for blow + up") called at C:/perl-5.34.0/site/lib/Test/More.pm line 511 Test::More::cmp_ok(Math::MPFR=SCALAR(0x86b998), "!=", Math::MP +FR=SCALAR(0x33cf88), "test for blow up") called at -e line 1 Cleaning up the CONTEXT stack... # Tests were run but no plan was declared and done_testing() was not s +een. # Looks like your test exited with 255 just after 1.
Note that the problem arises at C:/perl-5.34.0/site/lib/Test/Builder.pm line 1006.
It's not the test failure that's an issue - pretty obviously, that test should fail.
It's the ensuing fatal over-the-top blowup that's my main concern.
Here's the actual section of the Test::Builder code that blows up:
... unless($ok) { $self->$unoverload( \$got, \$expect ); if( $type =~ /^(eq|==)$/ ) { $self->_is_diag( $got, $type, $expect ); } elsif( $type =~ /^(ne|!=)$/ ) { no warnings; my $eq = ($got eq $expect || $got == $expect) ## LINE 1006 ...
The evaluation of $got eq $expect is bound to be interesting if $got is an object and the 'eq' operator has not been overloaded

Anyway ... my question is "what is the bug ?"
Is it that 'eq' overloading has not been provided ? (Providing such overloading successfully works around the problem.)
Is it that Test::More (or Test::Builder) should not be assuming that 'eq' has been overloaded ?
Or has Test::Builder tried (and failed) to account for this possibility. I'm looking at $self->$unoverload( \$got, \$expect ) in the above snippet from Builder.pm, and wondering what that might be intended to do.

With math objects, I've never felt the need to be able to compare $obj eq $str. If I need to do that I just make use of the overloaded stringification and compare "$obj" eq $str.

I'm interested to hear thoughts on what is the "correct" way to deal with this issue.

Cheers,
Rob

Replies are listed 'Best First'.
Re: The wonders of Test::More
by haukex (Bishop) on May 21, 2022 at 15:34 UTC
    I'm interested to hear thoughts on what is the "correct" way to deal with this issue.

    Though probably the most boring way of dealing with the issue, personally I would probably tend to simply use an ok test for a case like this... they way I see it, all the other Test::More functions are just sugar for ok.

    use warnings; use strict; use Test::More; my $exp = "two"; my $got = "Two"; ok $exp eq $got or diag explain [$exp, $got]; done_testing;
      ok $exp eq $got or diag explain [$exp, $got];

      Yeah, that's not a bad idea.
      I don't get much sense from "or diag explain [$exp, $got]" when the variables are Math::MPFR objects:
      C:\>perl -MMath::MPFR -MTest::More -e "$got=Math::MPFR->new(2); $exp= +Math::MPFR->new(2); ok($got != $exp) or diag explain [$exp, $got]; d +one_testing();" not ok 1 # Failed test at -e line 1. # [ # bless( do{\(my $o = 46472664)}, 'Math::MPFR' ), # bless( do{\(my $o = 46474008)}, 'Math::MPFR' ) # ] 1..1 # Looks like you failed 1 test of 1.
      but I could instead use something like or warn "$got == $expected" to provide a more meaningful diagnostic.

      There's probably quite a few places in my test scripts where this blow-up could occur, but the blow-up only happens with the overloaded "!=" operator, and then only if the test fails.

      Cheers,
      Rob
        I could instead use something like or warn "$got == $expected"

        That works, as well as simply naming the test e.g. "$exp eq $got", or I sometimes use Data::Dump's pp to get properly quoted stings (since explain doesn't do that for simple strings).

Re: The wonders of Test::More
by hv (Parson) on May 21, 2022 at 17:38 UTC

    That's deeply confusing.

    I suggest starting with use of Data::Dumper or possibly Devel::Peek to show what $got and $expect contain after the $unoverload() call.

    I suspect the problem here is that you're performing a failing numeric test on Math::MPFR objects which have stringify overload ('""'), but not numify ('0+'), and Test::Builder is mishandling that case at the point it wants to generate useful diagnostics for the failure.

    I don't have time to trace through it in detail right now, I'll try to get back to that.

      That's deeply confusing

      Indeed. To begin with, it was a case of TL;DR, so it took me a couple of weeks to summons the energy to actually look at it ;-)
      I initially just changed the condition form '!=' to '<' (which was actually a more appropriate condition anyway) and any failures were then reported normally.
      This issue is actually cropping up in a new module I'm putting together, and there's one particular function that I haven't yet got quite right ... hence the occasional failures.

      The diagnostics suggest that, after the $unoverload() call, both $got and $expect are still Math::MPFR objects, unaltered in any way.
      Devel::Peek::Dump() calls done before and after the $unoverload() call, confirm this.

      $unoverload is the string "_unoverload_num", which by the look of it (I'm guessing) disables the '0+' overloading.
      There is no '0+' overloadinging in Math::MPFR - I've never bothered with '0+' because I've never hit a case where it serves a useful purpose.
      AFAIK, when a Math::MPFR object is used in a numeric (unoverloaded) context, then the stringification of that Math::MPFR object is automatically used in numeric context. And that has always served just fine.
      (Mind you, this new module I'm playing with might well require '0+' overloading, as I doubt that the stringification will DWIM when used in numeric context.)

      I know there are aspects of overloading that I don't understand well ... and '0+' overloading could well be one of them.
      I think that Math::GMPq is the only module where I've overloaded '0+', and that was done simply because strings like '11/65537' don't DWIM when used in numeric context.
      For example, the condition ('11/65537' == '11/2') is TRUE .... and that doesn't DWIM well at all.

      Cheers,
      Rob

        Hugo here, I was typing up a response to this last night when my computer died stone dead with a strong whiff of ozone. I'll be offline for a few days until I can get that fixed.

        I found that the issue could easily be replicated with something roughly like the below. (This is from memory, I'm in a local hotel to write this.) Please verify and report to the Test::More bug queue.

        The problem appears to be occur when a) the '!=' test fails, b) the object(s) being tested have '!=' and '""' overloading but not '0+'. I have not checked if both arguments need to be objects, nor if other comparators are affected.

        use Test::More; package Ov { use overload ( '""' => sub { 'x' }, '!=' => sub { 0 }, }; }; cmp_ok(bless({}, 'Ov'), '!=', bless({}, 'Ov'), 'failing test'); done_testing();

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11144043]
Front-paged by Discipulus
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (4)
As of 2022-06-24 21:57 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My most frequent journeys are powered by:









    Results (80 votes). Check out past polls.

    Notices?