Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Precision
#25
That's why I suggested to @Chris to run the code he has if possible. Maybe put in "PRINT" statements to figure out what values are assigned to which variables. If this is too hard, run the code in QB64 IDE and put variable watch breakpoints. I don't know how to do that since I use the IDE only for code formatting and to execute very simple programs.

At first assume all variables are type _INTEGER64. If the "final" result is zero then at least one of the variables has to be assigned something useful. He has to test to make sure there are no negative integers involved, then in which case the positive range of _INTEGER64 could be doubled to _UNSIGNED _INTEGER64.

However, if he requires fractional parts anywhere in the computations then this explanation has to be thrown out the window. There's a lot of "INT" and "MOD" usage so we could almost assume only large integers are being involved. Assume because Chris is not telling us if he ever tried to run his code and how.
Reply


Messages In This Thread
Precision - by Chris - 11-22-2022, 08:43 PM
RE: Precision - by SMcNeill - 11-22-2022, 09:12 PM
RE: Precision - by mnrvovrfc - 11-22-2022, 09:14 PM
RE: Precision - by bplus - 11-22-2022, 09:15 PM
RE: Precision - by mnrvovrfc - 11-22-2022, 09:20 PM
RE: Precision - by Chris - 11-22-2022, 09:26 PM
RE: Precision - by Chris - 11-22-2022, 09:33 PM
RE: Precision - by Chris - 11-22-2022, 09:37 PM
RE: Precision - by Chris - 11-22-2022, 09:40 PM
RE: Precision - by mnrvovrfc - 11-22-2022, 09:41 PM
RE: Precision - by Kernelpanic - 11-23-2022, 12:58 AM
RE: Precision - by Chris - 11-23-2022, 04:22 PM
RE: Precision - by bplus - 11-23-2022, 04:30 PM
RE: Precision - by Chris - 11-23-2022, 05:28 PM
RE: Precision - by Kernelpanic - 11-23-2022, 07:45 PM
RE: Precision - by mnrvovrfc - 11-23-2022, 07:55 PM
RE: Precision - by Jack - 11-24-2022, 02:44 PM
RE: Precision - by Chris - 11-24-2022, 07:06 PM
RE: Precision - by Kernelpanic - 11-24-2022, 11:59 PM
RE: Precision - by Chris - 11-27-2022, 07:01 PM
RE: Precision - by Kernelpanic - 11-27-2022, 09:08 PM
RE: Precision - by Kernelpanic - 11-29-2022, 11:28 PM
RE: Precision - by bplus - 11-22-2022, 08:54 PM
RE: Precision - by Pete - 11-22-2022, 08:59 PM
Precision - by Chris - 11-22-2022, 09:00 PM
RE: Precision - by Chris - 11-22-2022, 09:04 PM
RE: Precision - by SMcNeill - 11-22-2022, 09:10 PM
RE: Precision - by Chris - 11-22-2022, 09:08 PM
RE: Precision - by Pete - 11-22-2022, 09:25 PM
RE: Precision - by bplus - 11-22-2022, 09:26 PM
RE: Precision - by bplus - 11-22-2022, 10:42 PM



Users browsing this thread: 5 Guest(s)