05-30-2024, 01:49 AM
(05-30-2024, 01:02 AM)TerryRitchie Wrote: For example. The result of x should be 1 but instead the value .9999999 is given:That's not a bug, that's just how floating point works. The number is stored in scientific notation with about 7 to 8 digits of precision (23 binary digits). If you take a look at the value you calculated in base 2, it's a repeating number of some kind, so it's impossible to store the exact value with only 23 binary digits. The fractional part gets cut off and you get a number close to the actual value of `2 / 454` but not the exact number. Once that has happened you end up with results like `.9999999` because your `f` is not exactly `1 / 227` when you multiply it by `227`.
f = 2 / 454
x = 227 * f
Jack's suggestion is a good one and might avoid all the error. The evaluation order matters with floating point because if you do the multiplication first you avoid using the `1 / 227` value, instead you do `227 * 2` which is easily stored as an exact number in the 23 binary digits of precision.