http://qs321.pair.com?node_id=266677

maksl has asked for the wisdom of the Perl Monks concerning the following question:

dear monks;

i have to deal with numeric limits in perl. not exactly of the size npiazza deals with, but i need to know my machine accuracy. it's knowledge is crucial in order to eventually trust the computed data. :)

i've come up with this rapid one-liner:
perl -e '$eps=1;while(1<1+$eps){$eps/=2;}print "$eps\n"'

please tell me if my assumption are nuts. on a debian x86 and alpha i get: 1.11022302462516e-16
number that is found on various googled site in relation with the "NAG Fortran Library" and its precesion (example).

e~(N)^(1/2)*eps
where eps would be the machine accuracy.
N the number of multiplications.
e would result in the supposed mean fault

thx for your advice maksl

Replies are listed 'Best First'.
Re: machine accuracy
by BrowserUk (Patriarch) on Jun 18, 2003 at 08:49 UTC

    Okay. That's a question I can understand:) Whether I can answer it is a different matter, but I'll have a go.

    If your numbers are in the range 0.0 to 1.0 (or possibly -1.0 to +1.0, but that being pedantic), then that number is significant. However, if your numbers get larger, in the range 1 .. 10, then the significant value is different. vis.

    perl58 -le "$eps = 1; while( 10 < 10 + $eps ){ $eps /= 2 ; } print $ep +s," 8.88178419700125e-016

    And when the number gets larger still, then it changes again.

    perl58 -le "$eps = 1; while( 100 < 100 + $eps ){ $eps /= 2 }print $eps +" 7.105427357601e-015

    And, of course, it works the other way too

    perl58 -le "$eps = 1; while( 0.1 < 0.1 + $eps ){ $eps /= 2 } print $ep +s" 6.93889390390723e-018
    Rather than carry on step-by-step, it's instructive to look at the output over a range of ranges.
    perl58 -e "for(-15..16) { $eps=1; while( qq[1E$_] < qq[1E$_] + $eps ) { $eps /= 2; } printf '%20.20f : %32.32f' . $/, qq[1E$_], $ +eps }" 0.00000000000000100000 : 0.00000000000000000000000000000010 0.00000000000001000000 : 0.00000000000000000000000000000039 0.00000000000010000000 : 0.00000000000000000000000000000631 0.00000000000100000000 : 0.00000000000000000000000000005049 0.00000000001000000000 : 0.00000000000000000000000000040390 0.00000000010000000000 : 0.00000000000000000000000000323117 0.00000000100000000000 : 0.00000000000000000000000005169879 0.00000001000000000000 : 0.00000000000000000000000082718061 0.00000010000000000000 : 0.00000000000000000000000661744490 0.00000100000000000000 : 0.00000000000000000000005293955920 0.00001000000000000000 : 0.00000000000000000000042351647363 0.00010000000000000000 : 0.00000000000000000000338813178902 0.00100000000000000000 : 0.00000000000000000010842021724855 0.01000000000000000000 : 0.00000000000000000043368086899420 0.10000000000000001000 : 0.00000000000000000693889390390723 1.00000000000000000000 : 0.00000000000000011102230246251565 10.00000000000000000000 : 0.00000000000000088817841970012523 100.00000000000000000000 : 0.00000000000000710542735760100190 1000.00000000000000000000 : 0.00000000000005684341886080801500 10000.00000000000000000000 : 0.00000000000090949470177292824000 100000.00000000000000000000 : 0.00000000000727595761418342590000 1000000.00000000000000000000 : 0.00000000005820766091346740700000 10000000.00000000000000000000 : 0.00000000093132257461547852000000 100000000.00000000000000000000 : 0.00000000745058059692382810000000 1000000000.00000000000000000000 : 0.00000005960464477539062500000000 10000000000.00000000000000000000 : 0.00000095367431640625000000000000 100000000000.00000000000000000000 : 0.00000762939453125000000000000000 1000000000000.00000000000000000000 : 0.0000610351562500000000000000000 +0 10000000000000.00000000000000000000 : 0.000976562500000000000000000000 +00 100000000000000.00000000000000000000 : 0.00781250000000000000000000000 +000 1000000000000000.00000000000000000000 : 0.0625000000000000000000000000 +0000 10000000000000000.00000000000000000000 : 1.000000000000000000000000000 +00000

    There are three things to notice here.

    1. The accuracy of perl floating point representation is not absolute in term of "accurate to 0.00000000000000001".

      It's accuracy is defined in terms of significant digits, rather than decimal places.

    2. Measuring this accuracy, is problematic.

      If you look at the left-hand number for 1e-1 is shows up as 0.10000000000000001000. This is because some decimal fractions simply cannot be represented accurately using a binary fraction. 0.1 is one such, an 0.9 is another.

      perl58 -le"printf '%32.32f', 0.9" 0.90000000000000002000000000000000
    3. Perl's floating point representation, is really the floating point representation (and accuracy) of the C compiler used to compile your version of perl.

      In turn, that accuracy is (probably) (one of the) IEEE floating point representation(s), which is the same as is used (in most cases) by your (and almost every) floating point processor. As such, you will probably get a similar level of accuracy (or inaccuracy) regardless of the language you use on your processor, unless the language uses it's own libraries to perform floating point math.

      If you need this type of accuracy, (if you were calculating the trajectories for spacecraft heading to Mars for example, or calculating the US national debt:), then perl also has Math::BigFloat libraries, which will allow you to calculate to a precision that is essentially limited only by the memory capacity of your computer. Which on an average 64 or 128MB machine is (probably) accurate enough to align your telescope to one edge of a human hair lying on the surface of Pluto (provided there are no trucks rumbling by within a thousands miles:) or to total up the number of atoms in the Earth, and our Solar System (and quite possibly all the stars we can see, but I won't guarentee that bit:). The penalty of course, is speed. If you need that kind of accuracy, you are going to have to be prepared to wait a while for your answers.

    For much more (and much more accurate:) information, you might like to read IEEE reals.

    In summary, perl's floats are accurate to approx 53-binary bits or approx. 16 significant (decimal) digits (not places).


    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller


      ...or to total up the number of atoms in the Earth, and our Solar System (and quite possibly all the stars we can see, but I won't guarentee that bit:).

      approximate number of stars in the universe: 10^22, approximate number of atoms in the universe: 10^83.

      these are rough and from memory, it's amazing that it's pretty easy to imagine numbers that are greater than all of the little-itty-bitty things that actually exist.

        Are those number accurate to 15 decimal places or 16:)

        it's amazing that it's pretty easy to imagine numbers that are greater than all of the little-itty-bitty things that actually exist.

        You've a better imagination than I:). The other day I was playing with Othello, and worked out that on an 8x8 board there are 3^63 possible positions. My first thought was that if I could represent each board position using 128-bits, I might be able to compile a complete compendium of all possible board positions--till I did the math!

        3^63 = 1.144561273^10 = 1_144_561_273_430_837_400_000_000_000_000

        But just how much storage would that require? Assuming 16 -bytes per board, if I packed them onto 73MB CD's with no overhead for the file system, then it would take

        59_810_339_179_409_363_000_000 cd's.

        A lot for sure. More than exist, or ever probably will, but it's still just a number. So I tried to take this a litte further, and the best I could come up with was that, assuming there are a nice round 6 billion people on the Earth, then it would require every person on Earth to have 1662 CD's, for every other person on the Earth, to be able to hold all those board permuations.

        Now, I'm not entirely sure that all the math in there is correct, it could be out by a magnitude or two, but it was sufficiently accurate to convince me to try a different approach:)

        I got in the habit of trying to visualise such numbers very early on. My last year of Junior school, age 11 (I've no idea how that translates into US style grades), we were learning about Napier's Bones, logarithms and stuff, and the teacher told us the old riddle about 1 grain of rice on the first square of a chess board and 2 on the second etc. He then had us weight 1 ounce of rice each and count the grains, and average the counts. Then we calculated the total number of grains, and from that the weight, and from that the volume, and discovered that it would fill the school hall that had a 20 ft high ceiling and was capable of seating 500 kids.

        Needless to say, I never forgot the experiment, nor the power of the visualisation, but trying to wrap my brain around just how much larger 10^83 is that 10^30 I cannot even begin to see a way:)


        Examine what is said, not who speaks.
        "Efficiency is intelligent laziness." -David Dunham
        "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller


      wooow BrowserUk++ thx for this extensive posting that really answers my question ;)
Re: machine accuracy
by tedrek (Pilgrim) on Jun 18, 2003 at 07:59 UTC

    you could just throw a use bignum; at the top of your script and not have to deal with numeric limits.

    as for that snippet, it seems right to me but I only have a slight clue about math. I believe perl uses double precision numbers on most platforms which means you have 53 bits of precision. I expect somebody will speak up if I'm wrong :)

Re: machine accuracy
by BrowserUk (Patriarch) on Jun 17, 2003 at 23:29 UTC

    I'm being a bit quick off the mark and should probably wait to see what other people make of your question, but could you point out what your question is please?

    You say "please tell me if my assumption are nuts", but I can't work out what your assumption are:)


    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller


      i'll clarify my question:
      is the number that the one-liner above is printing relevant for the accuracy of perl calculations?
      or i'm i completly wrong in assuming that number as important for numerical calculations in perl?
Re: machine accuracy
by TomDLux (Vicar) on Jun 18, 2003 at 09:27 UTC

    Take a look in your Numerical Methods text, or google for "floatingpoint number" mantissa, you'll get pages like read this.

    Basically, a floating point number consists of two portions:

    • The exponent determines the magnitude you can represent, how large or small a number can be. Can we represent the grains of sand in a sphere the size of the earth? in a sphere the size of the solar sytem? Can we approximate the number of electrons in a mile? in a light year? in a cubic parsec?
    • The mantissa specifies how precisely those numbers can be specified. Can you state the national deebt to the penny or only plus or minus a million? When you make a recipe, do you use a cup of water? or 1.0000 cups? Floating point numbers have a finite number of bits to represent the value. but if you are trying to represent 1.00 you might be off by as much as +/- 0.005 ... but trying to represent 1.00 * 10^6 your error boils down to +/- 5000. But in either case, you're within half a percent.

    --
    TTTATCGGTCGTTATATAGATGTTTGCA

Re: machine accuracy
by Maclir (Curate) on Jun 18, 2003 at 12:43 UTC
    Are you concerned about accuracy or precision? In general, accuracy is "how close to the correct value is my answer", while precision is "how many significant digits does my answer contain."

    What you are describing is precision - and as someone else mentioned, bignum could be your friend. However, once you start manipulating high precision values, you need to make sure your numerical methods do not reduce your accuracy. In the dim, dark past, I did an honours-level university applied math subject that dealt with how to structure calculations so that numerical rounding errors did not accumulate and destroy your accuracy. Sadly, I have forgotten much of it!

      thx Maclir for clarifying the definitions.
      i'm interested as the title and the little formula indicates in the accuracy, which as physician i would describe in analogy to a statistical random walk depending on the square root of how many times you walk(multiplicate) multiplicated with the inevitable machine inaccuracy.
      "how close to the correct value is my answer" permits some level of trust to the data, doesn't it?
      but please tell more if you can dig up :)