Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Extended KotD #25, #26, and #27: INTEGER DIVISION, CINT, and CLNG
#20
(05-04-2025, 10:38 PM)PhilOfPerth Wrote: @SMcNeill:
This stream has already achieved one major goal: to raise awareness of the unexpected results we may get when using Integers

Your final short snippet sums up everything (at least, for me)!

value < 0.5 rounds down.
value = 0.5 rounds to the nearest even integer.
value > 0.5 rounds up.

X.49999999999999 rounds down.
x.50000000000001 rounds up.

If that's what happens in every case - pos or neg - then the world is sweet!

Note that this is what we see in CINT, CLNG, _ROUND.  Other commands don't have this same behavior.

Any single, double, or float assigned to an integer obeys these same rules of rounding, as you can see from the below.  (Pay close attention to the .5 entries and see how one rounds up and the other rounds down, as they're both rounding to the closest even integer.

Code: (Select All)
Dim i As Integer
For x = 0 To 2 Step .1
    i = CInt(x)
    Print x, i,
    Select Case _ToStr$(i)
        Case Is < _ToStr$(x): Print "Round Down"
        Case Is = _ToStr$(x): Print "Equal"
        Case Is > _ToStr$(x): Print "Round up"
    End Select
Next

Also note that other rounding type commands don't obey this pattern or rule, such as INT (always rounds down), CEIL (always rounds up), FIX (truncates).   What I'm mentioning here is for CINT, CLNG, _ROUND, and direct assignment of floating point values to integer types.
Reply


Messages In This Thread
RE: Extended KotD #25, #26, and #27: INTEGER DIVISION, CINT, and CLNG - by SMcNeill - 05-04-2025, 11:29 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Extended KotD #23 and #24: _MIN and _MAX SMcNeill 0 586 01-16-2025, 09:51 AM
Last Post: SMcNeill
  Extended KotD #22: _MOUSEHIDDEN SMcNeill 0 593 01-14-2025, 12:29 AM
Last Post: SMcNeill
  Extended KotD #20: _NOTIFYPOPUP SMcNeill 4 1,298 12-05-2024, 01:55 AM
Last Post: Pete
  Extended KotD #21: _MIDISOUNDBANK a740g 8 2,705 08-30-2024, 07:32 PM
Last Post: Steffan-68
  Extended KotD #19: _UCHARPOS SMcNeill 1 869 06-26-2024, 12:54 AM
Last Post: SMcNeill

Forum Jump:


Users browsing this thread: 1 Guest(s)