Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Possible floating point error?
#5
The problem with floating point is just the nature of the beast.  You're *always* going to have degrees of imperfections in floating point math.

Let me illustrate why, using base-10, standard math.  Wink

Let's take two numbers and divide them together, to make a decimal value.  For illustration, I choose 1 and 3.

1 / 3 = 0.33333333... to however many digits of precision you want to go through.  SINGLE holds about 8 digits, DOUBLE 12, FLOAT 16 (or something like that).

So we take a SINGLE for 1 / 3...  the answer is 0.333333333 -- 8 digits of precision.

Now, let's take those 8 digits and multiply by 3: 0.33333333 * 3 = 0.99999999

It's not 1.   That loss of precision has now generated a basic Floating Point Imperfection for us.

And, at this point, you see some programs which try to be smart and round up the answer.  They basically say, "Hey!  0.99999999 is our max precision estimate for 1, so the answer is 1."

And then 0.33333333 * 3 = 1, and the world is good and everything makes sense!!

.....

At least it does until someone does math for the exact value 0.11111111 * 9.  They get 0.99999999, which we declared as "Meh, close enough to 1, so we call it 1," which is wrong.  


The basic root of the problem is we have no way to perfectly represent all fractions in decimal form.  

1/3 is impossible to represent perfectly in base-10.  

1/10 is impossible to represent perfectly in base-2 (binary).  

There's some simple rule to this, but I'm not certain I'm remembering it perfectly.  I think it's along the line of "ALL factors of the fractional denominator must be factors of the base, for perfect representation."

1/3, for example, fails in base-10 as 3 is not a factor of 10.
1/2, works in base 10, as 2 *is* a factor of 10.
1/4 works as 4 is just (2 * 2) and 2 is a factor of 10.
1/7 fails, as 7 is not a factor in 10.
Reply


Messages In This Thread
Possible floating point error? - by TerryRitchie - 05-30-2024, 01:02 AM
RE: Possible floating point error? - by Jack - 05-30-2024, 01:34 AM
RE: Possible floating point error? - by JRace - 05-30-2024, 01:48 AM
RE: Possible floating point error? - by SMcNeill - 05-30-2024, 02:30 AM
RE: Possible floating point error? - by SMcNeill - 05-30-2024, 03:31 AM



Users browsing this thread: 2 Guest(s)