Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
double to 2^53
#1
Ok, I know that the size of the mantissa of double is 53 bits, see if you can guess what values will be printed by this snippet

Code: (Select All)
Dim As Double x

x = 9007199254740992 ' 2^53
Print x
x = x + 1#
Print x
x = x + 1#
Print x
Print "================="
x = 9007199254740998
Print x
x = x + 1#
Print x
x = x + 1#
Print x
Print "================="
x = 9999999999999998
Print x
x = x + 1#
Print x
Reply
#2
53 bits isn't 2 ^ 53.

For example:  2 bits has a limit of 3 (&H11).  8 bits has a limit of 255 (&H11111111).

The limit is number of bits ^ 2 - 1.    (n ^ 2 - 1)
Reply
#3
hello SMcNeil
I think that you made typo, The limit is number of bits ^ 2 - 1.    (n ^ 2 - 1)
yo probably meant "The limit of number is 2^bits - 1.    (2^n  - 1)" or something like that
Reply
#4
Aye.  Sorry.   2 ^ n - 1.  I'm trying to get over Covid again, so my brain is half broken atm.  Apologies.
Reply
#5
I hope that you get well soon Smile
Reply




Users browsing this thread: 4 Guest(s)