Single v Double

[Pete]

@Jack

Involved semantics, I believe, when considering the sign...

True about truncated with FIX, I know, but I see the results appear to be rounded as follows:

PRINT FIX(3.8), FIX(-3.8) ' Results 3, -3. 3.8 to 3 is rounding down towards zero. -3.8 to -3 is rounding up towards zero. (-3 > -3.8)
PRINT INT(3.8), INT(-3.8) ' Results 3, -4. 3.8 to 3 is rounding down towards zero. -3.8 to -4 is rounding down towards zero. (-4 < -3.8)

Pete

[Jack]

I don't agree that Fix rounds up or down, it simply truncates the fractional part whereas int does round down

[Kernelpanic]

"Fix" is a rounding function like CINT & INT. It always returns the integer part of the argument.- With Fix one can also determine the decimal part.
Read more books folks! 

Code: (Select All)

Dim As Single a

Print Fix(23.67)
Print
Print Fix(-5.77)
Print
Print "----------------------"
Print
a = -27.007
Print a
a = Abs(a - Fix(a))
Print Using ".###"; a

[Pete]

@Jack

Sure it truncates, that's how it was set up to function, but the results are the same as if it rounded the way I described. Again, just semantics.

Pete

[Jack]

Kernelpanic, I don't need to read books in order to understand that if I cut a snake's head I didn't round it up or down, I simply truncated it's head off

[Pete]

I don't like being a round snakes, either.

I had another friend who was all about books. I tried so hard to get him to think outside the box, and one day he did. It was the day the movers came and moved all his books outside the box.

Pete

[Richard]

I'm damn sure you guys have explained this once before so if anyone could point me to the thread I'd appreciate it. Can't seem to come up with the one I kind of remember that addressed this, it could be on the other site. Anyway here's the simply code

Dim a As Double
Dim b As Single
a = .12345678
Print a
b = Abs(a * 10000000)
Print a; b


So my search is to find out what happened to the 7 in the difference between variable a and variable b

This topic is of a similar nature to an old thread: -

Possible error in INTEGER64 calculations
https://qb64forum.alephc.xyz/index.php?topic=4210.0

and my reply (last response to thread + sub-link (#183))'

Attached is the preliminary work in trying to establish the first transition point in going from Number Space Ordinary (NSO) where all positive integers are representable in _DOUBLE    TO      where the granularity is larger than 1 (i.e., resulting in "skipped" integers).

So, the above referred attachment (comparison table SINGLE vs DOUBLE vs FLOAT vs INTEGER64 etc.) among other things illustrates the "Moral": -

PRINT USING "may" give a more accurate representation of the compute result than PRINT only. 


Hope this reply here is of help explaining/illustrating.

[Kernelpanic]

I don't like being a round snakes, either.

I had another friend who was all about books. I tried so hard to get him to think outside the box, and one day he did. It was the day the movers came and moved all his books outside the box.

Pete

How can I look beyond the box if I don't know anything? And where do you get knowledge from? Your friend has probably moved with the boxes of books. Something is wrong with your story. . .

[Pete]

Well now you personally can tell all your friends you know someone who learned most of what he knows without the use of books. Other people, examples, trial and success, application, intuition, etc. I even got a lot of math answers correct by estimation and reverse engineering.

Pete