The suggestion by haukex in 11114564 can be followed more thoroughly. Dump each variable and partial step involved in the calculation to track what happens. This means $min, $squelch, ($min - $squelch), and so forth. It is probably worth printing them at a higher precision than the default to see more, e.g. printf "%.17f\n", ($min - $squelch) (although both those should be integers based on the original post).

Although, "should be integer" depends what you have done with the numbers. Does your code use $min or $squelch in a floating point context? This could cause them to be used as floats in later calcs. I'm not overly familiar with the internals, though, so cannot say what to expect with any confidence.