Re^2: OT: How much float precision needed for operation?


Perl: the Markov chain saw
	PerlMonks

Re^2: OT: How much float precision needed for operation?

by 5mi11er (Deacon)

on Sep 02, 2005 at 21:02 UTC ( [id://488788]=note: print w/replies, xml )

Need Help??

in reply to Re: OT: How much float precision needed for operation?
in thread OT: How much float precision needed for operation?

So, I think what your data says is:

To ensure an error of no more than plus/minus 0.1 we'd need ~7 digits to the right of the decimal point for y=10 and ~53 digits for y=100

Is this correct?

-Scott

Update: trammel, thank you very much for your excellent examples above.

So, for others who want to extend this to other functions:

If we take the derivative of the function we're working with, multiplied by a delta (difference), and know what our tolerance for error is, we can solve for the maximum delta that will be within our tolerance for error.
In math symbols, the process is:
given function F
find the derative -> dF
maxdelta = errortolerance/dF