in reply to Re^3: False positive on inequality comparison
in thread False positive on inequality comparison

For the record, I did try this as well, but it didn't help my Monday morning mind narrow anything down. Specifically, I found the same three differences. The binary representations didn't change between the time I read the values in and the time I compared them.

First two lines are reading files in from Excel sheets, second two are later in the script when the difference is found. I didn't include all three differences here, since they're all the same scenario. Different values read in from the files, no change to those values for the comparison:

read var_f in from change file: 10011100010100111000101110010000100001 +01000001010110011000000010 read var_f in from range file : 10101100010100111000101110010000100001 +01000001010110011000000010 var_f value in change hash : 10011100010100111000101110010000100001 +01000001010110011000000010 var_f value in range hash : 10101100010100111000101110010000100001 +01000001010110011000000010

I'm not familiar enough with floating point representation to say what the difference is (it's been years since I had to convert a binary to float, and unfortunately need to do actual work this morning rather than brush up on the process). I think this is a difference in exponents, but that wouldn't make a lot of sense to me with the rest of the binary value being identical.

Replies are listed 'Best First'.
Re^5: False positive on inequality comparison
by BrowserUk (Pope) on Apr 06, 2009 at 19:19 UTC

    Hm. When I unpack those two values, they are distinctly different:

    printf "%17.17f\n", unpack 'd', pack 'b64', '1001110001010011100010111001000010000101000001010110011000000010';; 181.01965800261112000 printf "%17.17f\n", unpack 'd', pack 'b64', '1010110001010011100010111001000010000101000001010110011000000010';; 181.01965800261101000 $first = unpack 'd', pack 'b64', '1001110001010011100010111001000010000101000001010110011000000010';; $second = unpack 'd', pack 'b64', '1010110001010011100010111001000010000101000001010110011000000010';; printf "%f %f ;> %17.17f\n", $first, $second, $first - $second;; 181.019658 181.019658 ;> 0.00000000000011369

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.