Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
_INTEGER64 calculation
#4
(07-25-2022, 10:12 PM)bplus Wrote: Works on my system! feels odd putting suffix on the ends of numbers.

It might feel a little odd at first, but once you think about the compilation process of your code, it makes perfect sense.

Let's take any old equation:  X = A + B + C

*HOW* do we get a value for X, from the above??

Our computer only adds two numbers at a time, so it'd tend to first add A + B, and store that value in a temporary variable.  That variable is then added to C, and repeating computer logic, it's stored in a temp variable. With no other math to do, that temp variable is then finally assigned to X.

-----------------------------------------

So, think of X = A + B

Step 1:  Add A + B, store the result in a temp variable.
Step 2: Assign X the value of that temp variable.

----------------------------------------

In an attempt to maximize speed and minimize memory usage, our compiler takes a shot in the dark at how much memory we need to do those calculations.  Generally speaking, most compilers will try and default to using 32-bit math registers for this type of calculation -- which in the case of 100,000 * 100,000 is beyond the limits of what a 32-bit variable can hold without overflowing.

By making one of our constants become an INT64 type, rather than an unsigned type where the compiler tries to guess at what works best, we're telling the compiler that it's going to need to fire up and use the 64-bit math registers and such, rather than the 32-bit ones.

Code: (Select All)
billion&& = 10000000000
Print billion&&
billionCalc&& = 100000 * 100000
Print billionCalc&&

l& = 10000000000
Print l&

As you can see from the simple code above, trying to store the same value in l& as we did in billion&&, overflows giving the exact same answer as our billionCalc&& had.

All you have to do is make certain that at least one  of the variables you're sending into the math registers is 64-bit, and you'll be certain to get back 64-bit results.  Make either of those 100000 become 100000&&, and the issue goes away.

Generally speaking, whenever dealing with anything which might result in greater than 32-bit values, you're going to want to make certain that all your constant values are 64-bit manually.  If you get in the habit of adding those type definition symbols, you'll save yourself overflow issues like this from ever happening with your code.  

It's a good habit to learn to acquire ASAP, IMHO.  Wink
Reply


Messages In This Thread
_INTEGER64 calculation - by vinceg2022 - 07-25-2022, 09:12 PM
RE: _INTEGER64 calculation - by SMcNeill - 07-25-2022, 09:33 PM
RE: _INTEGER64 calculation - by bplus - 07-25-2022, 10:12 PM
RE: _INTEGER64 calculation - by SMcNeill - 07-26-2022, 12:26 AM
RE: _INTEGER64 calculation - by vinceg2022 - 07-26-2022, 05:09 AM
RE: _INTEGER64 calculation - by aurel - 07-26-2022, 03:00 PM
RE: _INTEGER64 calculation - by Pete - 07-27-2022, 04:02 AM
RE: _INTEGER64 calculation - by bplus - 07-27-2022, 12:00 PM
RE: _INTEGER64 calculation - by aurel - 07-26-2022, 03:05 PM



Users browsing this thread: 5 Guest(s)