Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Possible floating point error?
#1
I have a routine that tracks multiple FPS rates within a master FPS rate. I noticed with certain values that it fails. My son actually created the routine for me a few years back. I showed him what was going on and we tracked it to a floating point error.

For example. The result of x should be 1 but instead the value .9999999 is given:

f = 2 / 454
x = 227 * f

(2 is the target FPS, 454 is the global FPS, 227 is the current global frame number)

I've already created a work around for this. My son's code is too precise for what I need. He's tracking the exact moment the frame changes, whereas I simply need to know the alternate frame number within the master FPS. His code:

Fraction = TargetFPS / GlobalFPS
x = CurrentGlobalFrameNumber * Fraction
IF INT(x) <> INT(x - Fraction) THEN ... (report that a frame change happened)

All I simply need is INT(x) for my purposes but the possibility of a floating point error may still exist in QB64. Am I correct in assuming this?
New to QB64pe? Visit the QB64 tutorial to get started.
QB64 Tutorial
Reply


Messages In This Thread
Possible floating point error? - by TerryRitchie - 05-30-2024, 01:02 AM
RE: Possible floating point error? - by Jack - 05-30-2024, 01:34 AM
RE: Possible floating point error? - by JRace - 05-30-2024, 01:48 AM
RE: Possible floating point error? - by SMcNeill - 05-30-2024, 02:30 AM
RE: Possible floating point error? - by SMcNeill - 05-30-2024, 03:31 AM



Users browsing this thread: 1 Guest(s)