Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re^5: Avoiding perl's Atof when assigning floating point values

by syphilis (Archbishop)
on Jul 29, 2018 at 00:59 UTC ( [id://1219423]=note: print w/replies, xml ) Need Help??


in reply to Re^4: Avoiding perl's Atof when assigning floating point values
in thread Avoiding perl's Atof when assigning floating point values

UPDATE: Since writing this post, I've managed to reproduce the lib/locale.t failure on my Ubuntu-16.04 box by switching locale to LANG=de_DE.UTF-8.
I now just need to make sense of what I'm seeing.

The failures might be related to the fact the my locale (like some European ones) uses a comma (,) as decimal separator, unlike a dot (.) as expected by "C" locale

Then I wonder if I might be able to strike the same issues on my Ubuntu-16.04 box by messing with its locale settings. I'll have to at least give that a try.
I've no experience with messing with locale settings - but I expect that google has lots of advice to offer on how to do this.

I'm sure that my patches affect only the values that are assigned - they change nothing about the way that values are displayed.
If C's strtod() doesn't honor the locale when parsing the input string, then that would cause breakage. However, in such a case I would expect more extensive breakage than just a lib/locale.t failure.
And, of course, such a perl would have no business in defining Perl_strtod in the first place - and the default that allows the defining of $Config{d_strtod} ought to be changed.

Could you test one more thing for me, please ?
When (if) you have the time, could you rebuild that source with the addition of -Ud_strtod to the configure args. That is:
make distclean sh Configure -des -Dusedevel -Duse64bitall -Ud_strtod && make && make +test
I'm hopeful that will allow the lib/locale.t tests to pass without introducing more breakage elsewhere.
Thank you very much for the trouble you've taken and the information you've provided.

Cheers,
Rob

Replies are listed 'Best First'.
Re^6: Avoiding perl's Atof when assigning floating point values
by syphilis (Archbishop) on Jul 29, 2018 at 12:56 UTC
    I've been a little despondent about the lib/locale.t failures ever since I became aware of them a few days ago.
    But, tonight, I think I've found the correct way to deal with them.
    Under make test there seems to be only the one failing test in lib/locale.t, but there's an additional 4 tests that fail if one runs ./perl -I./lib -T locale.t
    There's no problem with those 5 tests on quadmath builds, irrespective of locale ... why is that ? ... it's because they're skipped on quadmath builds.

    It then dawned upon me that if it's ok for quadmath builds to skip those tests, then it's ok for my patched approach to also skip them. (Like the quadmath builds, my approach also assigns NVs using the underlying C compiler via Perl_strtod.)
    Having just modified lib/locale.t to skip those tests whenever Perl_strtod is defined, I find that all tests now pass !!

    I just need to provide patches against current blead, and hope that I can get Karl and/or Jim to create another smoke-me branch for more testing.
    If I can just get those patches into blead .....

    Cheers,
    Rob
      It then dawned upon me that if it's ok for quadmath builds to skip those tests, then it's ok for my patched approach to also skip them

      However, quadmath's inability to pass those tests didn't change any pre-existing behaviour, whereas my patches have changed behaviour - and that might not be greeted so warmly.
      I probably ought at least find out exactly what that change of behaviour is - though it's as clear as mud to me at the moment.
      It would be great if anyone has some clear code that demonstrates what has actually changed.
      In the meantime, it's back to perusing the perllocale docs and lib/locale.t. (Is there some other documentation that might be helpful to me ?)

      Cheers,
      Rob
        I probably ought at least find out exactly what that change of behaviour is

        Here's the script that demonstrates that change of behaviour:
        use warnings; use locale; $m = "3.14e+9" + 0; $n = "3,14e+9" + 0; print "\$m is $m\n"; print "\$n is $n\n";
        On current bleadperl (and recent "double" and "long double" perls) it outputs:
        $m is 3140000000 $n is 3140000000
        But when I patch bleadperl to assign with Perl_strtod (or run that script on a quadmath build of perl) it outputs:
        $m is 3140000000 $n is 3
        AFAICT, it pertains solely to the numification of strings. When values are assigned as barewords, there's no difference between the behaviour of the various builds.
        And I think it comes into play only if the locale radix character is a comma.
        My next step is to find out whether this needs to be fixed before my proposed patches can be applied.

        Cheers,
        Rob
Re^6: Avoiding perl's Atof when assigning floating point values
by aitap (Curate) on Jul 29, 2018 at 12:19 UTC

    I've managed to reproduce the lib/locale.t failure on my Ubuntu-16.04 box by switching locale to LANG=de_DE.UTF-8.
    I wonder if it's possible to force a particular LC_NUMERIC which has decimal_point   "<U002C>" in /usr/share/i18n/locales/$locale for the lib/locale.t test set, while keeping it C for other categories as a way of verifying the cause of the failures.

    make distclean sh Configure -des -Dusedevel -Duse64bitall -Ud_strtod && make && make +test
    lib/locale.t passes, but other tests don't:
    Failed 2 tests out of 2458, 99.92% okay. ../ext/POSIX/t/posix.t run/locale.t
    But that's not too mysterious: In particular, with those patches: all tests pass.

      Failed 2 tests out of 2458, 99.92% okay. ../ext/POSIX/t/posix.t run/locale.t

      Yes, the first thing I tried when I discovered I could reproduce the failures was to test a build configured with -Ud_strtod.
      I got the same results as you.

      I apologise for not having tried a bit harder to reproduce the lib/locale.t failure.
      I assumed that Carlos' smoker would be using the same locale setting for all of the systems that it tested. That is, I assumed that the reason that some systems passed and others didn't was because of some difference in the systems.
      But, of course, it was the difference in the locale settings. It was a bad assumption on my part. (They always are.)

      As I mentioned in my last post, I'm confident that the correct fix is to modify lib/locale.t and that's the solution I'll be pushing when I get things organized.
      Here's the patch:
      --- git/bleadperl/lib/locale.t 2018-07-15 10:33:46 +1000 +++ locale.t 2018-07-29 22:21:25 +1000 @@ -2167,8 +2167,8 @@ my $first_c_test = $locales_test_number; $test_names{++$locales_test_number} = 'Verify that a different lo +cale radix works when doing "==" with a constant'; - if ($Config{usequadmath}) { - print "# Skip: no locale radix with usequadmath ($test_names{ +$locales_test_number})\n"; + if ($Config{usequadmath} || ($Config{uselongdouble} && $Config{d_ +strtold}) || (!$Config{uselongdouble} && $Config{d_strtod})) { + print "# Skip: no locale radix with Perl_strtod ($test_names{ +$locales_test_number})\n"; report_result($Locale, $locales_test_number, 1); } else { report_result($Locale, $locales_test_number, $ok3); @@ -2176,8 +2176,8 @@ } $test_names{++$locales_test_number} = 'Verify that a different lo +cale radix works when doing "==" with a scalar'; - if ($Config{usequadmath}) { - print "# Skip: no locale radix with usequadmath ($test_names{ +$locales_test_number})\n"; + if ($Config{usequadmath} || ($Config{uselongdouble} && $Config{d_ +strtold}) || (!$Config{uselongdouble} && $Config{d_strtod})) { + print "# Skip: no locale radix with Perl_strtod ($test_names{ +$locales_test_number})\n"; report_result($Locale, $locales_test_number, 1); } else { report_result($Locale, $locales_test_number, $ok4); @@ -2198,8 +2198,8 @@ $test_names{$locales_test_number} = 'Verify that "==" with a scal +ar still works in inner no locale'; $test_names{++$locales_test_number} = 'Verify that "==" with a sc +alar and an intervening sprintf still works in inner no locale'; - if ($Config{usequadmath}) { - print "# Skip: no locale radix with usequadmath ($test_names{ +$locales_test_number})\n"; + if ($Config{usequadmath} || ($Config{uselongdouble} && $Config{d_ +strtold}) || (!$Config{uselongdouble} && $Config{d_strtod})) { + print "# Skip: no locale radix with Perl_strtod ($test_names{ +$locales_test_number})\n"; report_result($Locale, $locales_test_number, 1); } else { report_result($Locale, $locales_test_number, $ok8); @@ -2218,8 +2218,8 @@ $problematical_tests{$locales_test_number} = 1; $test_names{++$locales_test_number} = 'Verify that after a no-loc +ale block, a different locale radix still works when doing "==" with +a scalar and an intervening sprintf'; - if ($Config{usequadmath}) { - print "# Skip: no locale radix with usequadmath ($test_names{ +$locales_test_number})\n"; + if ($Config{usequadmath} || ($Config{uselongdouble} && $Config{d_ +strtold}) || (!$Config{uselongdouble} && $Config{d_strtod})) { + print "# Skip: no locale radix with Perl_strtod ($test_names{ +$locales_test_number})\n"; report_result($Locale, $locales_test_number, 1); } else { report_result($Locale, $locales_test_number, $ok11); @@ -2465,8 +2465,8 @@ } } - if ($Config{usequadmath}) { - print "# Skip: no locale radix with usequadmath ($Locale) +\n"; + if ($Config{usequadmath} || ($Config{uselongdouble} && $Confi +g{d_strtold}) || (!$Config{uselongdouble} && $Config{d_strtod})) { + print "# Skip: no locale radix with Perl_strtod ($Locale) +\n"; report_result($Locale, $locales_test_number, 1); } else { report_result($Locale, $locales_test_number, @f == 0);
      (Safest to avoid line wrapping by electing to "download")

      If you apply that patch to the lib/locale.t that shipped with blead_patched.tar.gz then all tests should (hopefully) pass.

      Thanks again for the help you've provided. I much appreciate it.

      Cheers,
      Rob
      UPDATE: Patches for t/run/locale.t and ext/POSIX/t/posix.t have been submitted to perlbug - see https://rt.perl.org/Ticket/Display.html?id=133417
      UPDATE 2: Those patches to t/run/locale.t and ext/POSIX/t/posix.t have now been applied to bleadperl.

      Oh ... I failed to specifically thank you for providing those 2 patches.
      I'll certainly be including them as I think that -Ud_strtod builds should be supported.
      With current perl source there's no point in building with -Ud_strtod (or -Ud_strtold) but with the patches I'm proposing, those switches provide the simplest way to resort to using perl's Atof for assigning floating point values.
      I don't expect that anyone in their right mind would want to opt for using perl's Atof, but I think the option should remain and the test suite should pass all tests.

      Cheers,
      Rob

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1219423]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others admiring the Monastery: (6)
As of 2024-03-28 19:54 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found