Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Variable stuck on 24 bits
#10
(09-26-2023, 02:16 AM)SMcNeill Wrote:
(09-26-2023, 12:58 AM)DimColby Wrote:
(09-26-2023, 12:25 AM)SMcNeill Wrote: 1.677722E+07 is floating point, and you're at max single digit precision.

16777220 <-- this is what the value above translates to.
16777221 <-- add one
1.677722E+07 <-- and the above still rounds to the same value.

You're doing floating point math, not integer addition.
Thank you, I think that fixed it. It turns out that declaring it with a DIM didn't really do it, but not declaring it and using the & suffix did. 

Those two things must not be exactly the same.

Can you share some code with the glitch?  & suffix is the same as DIM AS LONG.  I imagine you're seeing an issue with your variable's scope.
I see what I did. I was misusing the DIM...AS... statement with a variable list. It's something I remember from way back, using that with lists of variables can give unexpected results and it's safer and no more difficult to declare variables one at a time. Thanks again!
Reply


Messages In This Thread
Variable stuck on 24 bits - by DimColby - 09-25-2023, 11:03 PM
RE: Variable stuck on 24 bits - by mnrvovrfc - 09-26-2023, 12:04 AM
RE: Variable stuck on 24 bits - by bplus - 09-26-2023, 12:11 AM
RE: Variable stuck on 24 bits - by DimColby - 09-26-2023, 01:28 AM
RE: Variable stuck on 24 bits - by SMcNeill - 09-26-2023, 12:25 AM
RE: Variable stuck on 24 bits - by DimColby - 09-26-2023, 12:58 AM
RE: Variable stuck on 24 bits - by SMcNeill - 09-26-2023, 02:16 AM
RE: Variable stuck on 24 bits - by DimColby - 09-26-2023, 06:31 AM
RE: Variable stuck on 24 bits - by SMcNeill - 09-26-2023, 12:31 AM
RE: Variable stuck on 24 bits - by SMcNeill - 09-26-2023, 02:20 AM
RE: Variable stuck on 24 bits - by bplus - 09-26-2023, 09:11 AM



Users browsing this thread: 4 Guest(s)