in reply to Re: int() function
in thread int() function
The mantissa of a double has 52 bits°, you need 50 bits to encode 15 decimal digits ( 2**10 =1024, so rule of thumb 10 bits for 3 decimals)
So the last two bits are used for error correction, since they don't suffice to encode another decimal digit.
Is it a good solution? Well I'd say appropriate for the era and not "for a dubious reason".
Are there better approaches? Sure, but they are harder to implement and would have been quite slow back in the days.
From my experience, >95% of the problems arise from calculating with currencies and the solution is obvious, calculate with integer cents and move the point only for output.
Cheers Rolf
(addicted to the Perl Programming Language :)
Wikisyntax for the Monastery
°) WP actually says 53 by using a spare bit redundancy , but tl;dr and this still fails to encode 16 decimals.
|
---|
Replies are listed 'Best First'. | |
---|---|
Re^3: int() function
by syphilis (Archbishop) on Oct 25, 2020 at 00:46 UTC | |
by LanX (Saint) on Oct 25, 2020 at 12:09 UTC | |
by syphilis (Archbishop) on Oct 25, 2020 at 13:05 UTC | |
by LanX (Saint) on Oct 25, 2020 at 18:21 UTC | |
by syphilis (Archbishop) on Oct 26, 2020 at 02:28 UTC | |
| |
Re^3: int() function
by ikegami (Patriarch) on Oct 30, 2020 at 12:11 UTC |