Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Steve's Programming Challenge: Weights and Measures
#7
(08-16-2024, 12:24 AM)dano Wrote: I wrote my program to check all combinations of the above and discovered that the .1 LB could get close, but not exact:

Exactly. And that's the error which we see in floating point math. We can get close to representing some values, but never exact. Wink

That *close but not exact* is the cause of floating point errors that we see when doing math. It's simply impossible to perfectly represent some values in binary-numbers, just as it's impossible to perfectly represent some numbers in standard base-10 values.

What is 1/3 as a decimal?

0.33333333333333333333333333333333333333333333333333333333333(keep holding down that 3 key... you haven't got it perfect yet...)

We can't perfectly represent 1/3 in decimal.

We also can't perfectly represent 1/10 in binary. We get "close, but not exact", and that "close" is where our math errors and rounding errors occur over time in our programs.
Reply


Messages In This Thread
RE: Steve's Programming Challenge: Weights and Measures - by SMcNeill - 08-16-2024, 12:35 AM



Users browsing this thread: 21 Guest(s)