10-31-2022, 05:42 PM
(10-31-2022, 05:32 PM)bplus Wrote: .1 decimal =
.00011 repeat 0011 forever (can't draw a bar over top of 0011)
like 1/3 = .3 with bar ontop of 3.
Another question solved with string math! ;-))
Just having fun, I know we can't do "bars on top" with our binary computers.
Question solved -- it's *impossible* to represent 0.1 *perfectly* in floating point values!
Which goes to also answer your question of: "Why y not 35.5100000000000000?"
It's just one of those impossible representations in binary math. As you said, we can't *truly* represent 1/3 in decimal math (at least not without the use of a repeater bar), and we can't represent the value of 35.51 perfectly either. It's an imprecise value and thus gives imprecision errors when used. It's just the nature of floating point values and imprecise fractions.
A much better solution is to keep everything as INTEGER values and then divide by 100 only at the display time for the end user. Instead of working with 35.51, work with 3551. Count pennies, not dollars, and you can avoid a vast majority of these problems.
