Single v Double

[Dimster]

I'm damn sure you guys have explained this once before so if anyone could point me to the thread I'd appreciate it. Can't seem to come up with the one I kind of remember that addressed this, it could be on the other site. Anyway here's the simply code

Dim a As Double
Dim b As Single
a = .12345678
Print a
b = Abs(a * 10000000)
Print a; b


So my search is to find out what happened to the 7 in the difference between variable a and variable b

[Jack]

simple, the digit 7 is rounded-up to 8, try this
Dim a As Double
Dim b As Single
a = .12345678
Print a
b = Fix(a * 10000000)
Print a; b

[Pete]

Just think of computers as women. If you push the wrong buttons, you can end up turning them off but while they are up they can't do math.

Pete

[mnrvovrfc]

"SINGLE" type is like a holdover of 8-bit computers such as the Radio Shack TRS-80 Color Computer 2 which had only two types: "numeric" and string. It didn't care if the "numeric" was an integer, and it had five or six digits of floating-point precision. Use "SINGLE" when the programmer must have fractional quantities but the accuracy is coarse, such as small currency values.

"DOUBLE" was invented to try to do something about "SINGLE" not going far enough, yet it is dwarfed by types subsequently invented while we made the transition from 16-bit to 32-bit and then to 64-bit. M$ BASIC PDS v7.1 and VBDOS had "CURRENCY" type but that was somewhat confusing; internally it was stored as "LONG" (or maybe 64-bit integer?) but was shown to the user having four digits of fraction precision.

"_FLOAT" was supposed to be more precise than "DOUBLE" with very large and very small numbers. However, up to 17 places for an integer, "_INTEGER64" is better. As was looked up by Steve, numbers can never be accurate like scientists want because the internal system to store the numbers is base-2, at least for a general purpose computer.

Some people are dissatisfied with the accuracy from the existing floating-point types not being good enough and that's why we're seeing a lot of "string arithmetic" libraries going around.

For example, might want to look at this thread:

https://qb64phoenix.com/forum/showthread...11#pid8811

The way floating-point numbers are stored is necessary in other programming languages to have a value which is like negative infinity or "Not A Number". Then the powers that be decided there should have been "different" NaNs... it's better not to think much about it. In M$ BASIC, "SQR(-1)" always gave out error #5. In some other programming language it could return NaN. There has to be a value that represents it although it's "Not A Number", get it? BTW this is told apart from "complex number" library, this is just an example of how BASIC could react to an "illegal" value compared to a different programming language.

[Jack]

mnrvovrfc
I don't agree with all you said, a single can hold 7 decimal digits and _Float can hold 19 decimal digits, _Float should hold a 64-bit integer accurately

[Dimster]

I was thinking the use of ABS would have prevented the rounding effect from double to single, but even if it the formula is written  b = (abs(a))*10000000 , Abs isn't working but 

b = Fix(A*10000000) does the trick. 

Thanks gang.

[Pete]

ABS only reports the positive value of a positive or negative number.

INT rounds any number down.

FIX rounds positive numbers down, and negative numbers up.

PETE uses string math so he doesn't have to put up with this ....

Pete

[Jack]

Pete
you have it wrong, int will round-down, fix simply truncates whether positive or negative, it's the same
int(-1234567.8) -> -1234568 , -1234568 < -1234567

[Kernelpanic]

According to the description: Abs is often used, for example in approximation algorithms, where it is important to stop the calculation above a certain level of accuracy. It doesn't matter whether the calculated value is larger or smaller than the comparison value.

Code: (Select All)

Dim As Double a, c, d
Dim b As Single

a = .12345678
Print a
b = Abs(a * 10000000)
Print a; b

Print
c = Abs(a * 10000000)
Print c

Print
c = .12345678
d = Abs(c * 10000000)
Print "c = ";: Print c
Print "d = ";: Print d

[mnrvovrfc]

According to the description: Abs is often used, for example in approximation algorithms, where it is important to stop the calculation above a certain level of accuracy. It doesn't matter whether the calculated value is larger or smaller than the comparison value.

Usually it's for very small numbers that cannot be displayed by the system such as 0.000001. Something like that is multiplied by a million along with something else, and then the difference of the two is checked to see if it's close to zero enough.

Pete
you have it wrong, int will round-down, fix simply truncates whether positive or negative, it's the same
int(-1234567.8) -> -1234568 , -1234568 < -1234567

Pete gave us the technical definition of your explanation.

https://qb64phoenix.com/qb64wiki/index.php/FIX