Scientific Notation

[james2464]

I'm curious about why this shows up so often as the answer for a number that is nearly zero.

For example,

a = 1 / 256
Print a

(I get 3.90625E-03)

Is this just a matter of using an appropriate variable type?

[BSpinoza]

It's a problem of the  IEEE Standard for Floating-Point Arithmetic (IEEE 754):
https://en.wikipedia.org/wiki/IEEE_754

Try PRINT USING!

Code: (Select All)

PRINT USING "####.##########",a

[james2464]

It's a problem of the  IEEE Standard for Floating-Point Arithmetic (IEEE 754):
https://en.wikipedia.org/wiki/IEEE_754

Try PRINT USING!

Code: (Select All)

PRINT USING "####.##########",a

Thanks!  That works great.

[Pete]

Actually, I think you need a semi colon, not a comma there...

Code: (Select All)

PRINT USING "####.##########"; a

Anyway, there are a lot of these floating point bombs to contend with.Here's two simple addition ones that differ with variable types.

Code: (Select All)

DIM a AS DOUBLE
FOR i = 1 TO 10
    a = a + .01
    PRINT a ' Bombs at iteration 7.
NEXT

DIM b as SINGLE
FOR i = 1 TO 10
    b = b + .01
    PRINT b ' Bombs at iteration 10.
NEXT

PRINT USING and FIX() are a couple of workarounds but for better reliability across a large number spectrum you need to consider using something like my string math routine, I believe Mark and Steve have ones of their own, too, or Jack's excellent numeric dec-float routine. Of course you need to call these as libraries and add them with an $INCLUDE statement or just get comfortable with a sub addition that consists of several hundreds of lines.

Pete

[james2464]

Thanks.

In my case, I'm displaying radians and every time it got near zero it changed to scientific notation. PRINT USING "#.##";a works perfectly.

[bplus]

Actually, I think you need a semi colon, not a comma there...

Code: (Select All)

PRINT USING "####.##########"; a

Anyway, there are a lot of these floating point bombs to contend with.Here's two simple addition ones that differ with variable types.

Code: (Select All)

DIM a AS DOUBLE
FOR i = 1 TO 10
    a = a + .01
    PRINT a ' Bombs at iteration 7.
NEXT

DIM b as SINGLE
FOR i = 1 TO 10
    b = b + .01
    PRINT b ' Bombs at iteration 10.
NEXT

PRINT USING and FIX() are a couple of workarounds but for better reliability across a large number spectrum you need to consider using something like my string math routine, I believe Mark and Steve have ones of their own, too, or Jack's excellent numeric dec-float routine. Of course you need to call these as libraries and add them with an $INCLUDE statement or just get comfortable with a sub addition that consists of several hundreds of lines.

Pete

Yeah but expensive in LOC and there always seems to be something else you need to fix!
Here we need a rounder and that usually needs to convert number back from string.

Code: (Select All)

_Title "N2S$ Test Steve's converter for number display" ' b+ 2022-11-10

Dim a As Double
For i = 1 To 10
    a = a + .01
    Print a; Tab(40); N2S$(Str$(a)) ' Bombs at iteration 7.
Next

Dim b As Single
For i = 1 To 10
    b = b + .01
    Print b; Tab(40); N2S$(Str$(b)) ' Bombs at iteration 10.
Next

' call N2S$(Str$(YourNumberOfAnyType))
Function N2S$ (EXP$) 'remove scientific Notation to String (~40 LOC)
    'SMcNeill Jan 7, 2020 ref: https://www.qb64.org/forum/index.php?topic=1555.msg112989#msg112989
    'Last Function in code marked Best Answer (removed debug comments and blank lines added these 2 lines.)
    ReDim t$, sign$, l$, r$, r&&
    ReDim dp As Long, dm As Long, ep As Long, em As Long, check1 As Long, l As Long, i As Long
    t$ = LTrim$(RTrim$(EXP$))
    If Left$(t$, 1) = "-" Or Left$(t$, 1) = "N" Then sign$ = "-": t$ = Mid$(t$, 2)
    dp = InStr(t$, "D+"): dm = InStr(t$, "D-")
    ep = InStr(t$, "E+"): em = InStr(t$, "E-")
    check1 = Sgn(dp) + Sgn(dm) + Sgn(ep) + Sgn(em)
    If check1 < 1 Or check1 > 1 Then N2S = _Trim$(EXP$): Exit Function 'If no scientic notation is found, or if we find more than 1 type, it's not SN!
    Select Case l 'l now tells us where the SN starts at.
        Case Is < dp: l = dp
        Case Is < dm: l = dm
        Case Is < ep: l = ep
        Case Is < em: l = em
    End Select
    l$ = Left$(t$, l - 1) 'The left of the SN
    r$ = Mid$(t$, l + 1): r&& = Val(r$) 'The right of the SN, turned into a workable long
    If InStr(l$, ".") Then 'Location of the decimal, if any
        If r&& > 0 Then
            r&& = r&& - Len(l$) + 2
        Else
            r&& = r&& + 1
        End If
        l$ = Left$(l$, 1) + Mid$(l$, 3)
    End If
    Select Case r&&
        Case 0 'what the heck? We solved it already?
            'l$ = l$
        Case Is < 0
            For i = 1 To -r&&
                l$ = "0" + l$
            Next
            l$ = "." + l$
        Case Else
            For i = 1 To r&&
                l$ = l$ + "0"
            Next
            l$ = l$
    End Select
    N2S$ = sign$ + l$
End Function

Another case solved by string math!

[bplus]

OK see what breaks this?
(I did the rounding first then called Steves N2S$ then chopped to digits:

Code: (Select All)

_Title "N2S$ Test Steve's converter for number display" ' b+ 2022-11-10
Print "Type Double:"
Dim a As Double
For i = 1 To 10
    a = a + .000000001
    Print a; Tab(40); roundDP$(a, 10) ' Bombs at iteration 7.
Next
Print "Type Single:"
Dim b As Single
For i = 1 To 10
    b = b + .000000001
    Print b; Tab(40); roundDP$(b, 10) ' Bombs at iteration 10.
Next

' this function needs N2S$ watch number type??
Function roundDP$ (num, digits) 'unsimplified  2021-09-16
    Dim s$, dot
    s$ = N2S$(Str$(num + (Sgn(num) * .5) * 10 ^ -digits))
    dot = InStr(s$, ".")
    If dot Then roundDP$ = Mid$(s$, 1, dot + digits) Else roundDP$ = s$
End Function

Function N2S$ (EXP$) 'remove scientific Notation to String (~40 LOC)
    'SMcNeill Jan 7, 2020 ref: https://www.qb64.org/forum/index.php?topic=1555.msg112989#msg112989
    'Last Function in code marked Best Answer (removed debug comments and blank lines added these 2 lines.)
    ReDim t$, sign$, l$, r$, r&&
    ReDim dp As Long, dm As Long, ep As Long, em As Long, check1 As Long, l As Long, i As Long
    t$ = LTrim$(RTrim$(EXP$))
    If Left$(t$, 1) = "-" Or Left$(t$, 1) = "N" Then sign$ = "-": t$ = Mid$(t$, 2)
    dp = InStr(t$, "D+"): dm = InStr(t$, "D-")
    ep = InStr(t$, "E+"): em = InStr(t$, "E-")
    check1 = Sgn(dp) + Sgn(dm) + Sgn(ep) + Sgn(em)
    If check1 < 1 Or check1 > 1 Then N2S = _Trim$(EXP$): Exit Function 'If no scientic notation is found, or if we find more than 1 type, it's not SN!
    Select Case l 'l now tells us where the SN starts at.
        Case Is < dp: l = dp
        Case Is < dm: l = dm
        Case Is < ep: l = ep
        Case Is < em: l = em
    End Select
    l$ = Left$(t$, l - 1) 'The left of the SN
    r$ = Mid$(t$, l + 1): r&& = Val(r$) 'The right of the SN, turned into a workable long
    If InStr(l$, ".") Then 'Location of the decimal, if any
        If r&& > 0 Then
            r&& = r&& - Len(l$) + 2
        Else
            r&& = r&& + 1
        End If
        l$ = Left$(l$, 1) + Mid$(l$, 3)
    End If
    Select Case r&&
        Case 0 'what the heck? We solved it already?
            'l$ = l$
        Case Is < 0
            For i = 1 To -r&&
                l$ = "0" + l$
            Next
            l$ = "." + l$
        Case Else
            For i = 1 To r&&
                l$ = l$ + "0"
            Next
            l$ = l$
    End Select
    N2S$ = sign$ + l$
End Function

[Pete]

Yep. My string math routine also includes a SI to numeric converter I coded. For my old office routines, I used to put pennies to dollars calculate and divide with a rounding formula that prevented crap from happening to monetary transactions. I think I tested it up to a million dollars, no problems.

Pete

[mnrvovrfc]

Yeah but expensive in LOC and there always seems to be something else you need to fix!
Here we need a rounder and that usually needs to convert number back from string.

Code: (Select All)

_Title "N2S$ Test Steve's converter for number display" ' b+ 2022-11-10

Dim a As Double
For i = 1 To 10
    a = a + .01
    Print a; Tab(40); N2S$(Str$(a)) ' Bombs at iteration 7.
Next

Dim b As Single
For i = 1 To 10
    b = b + .01
    Print b; Tab(40); N2S$(Str$(b)) ' Bombs at iteration 10.
Next

' call N2S$(Str$(YourNumberOfAnyType))
Function N2S$ (EXP$) 'remove scientific Notation to String (~40 LOC)
    'SMcNeill Jan 7, 2020 ref: https://www.qb64.org/forum/index.php?topic=1555.msg112989#msg112989
    'Last Function in code marked Best Answer (removed debug comments and blank lines added these 2 lines.)
    ReDim t$, sign$, l$, r$, r&&
    ReDim dp As Long, dm As Long, ep As Long, em As Long, check1 As Long, l As Long, i As Long
    t$ = LTrim$(RTrim$(EXP$))
    If Left$(t$, 1) = "-" Or Left$(t$, 1) = "N" Then sign$ = "-": t$ = Mid$(t$, 2)
    dp = InStr(t$, "D+"): dm = InStr(t$, "D-")
    ep = InStr(t$, "E+"): em = InStr(t$, "E-")
    check1 = Sgn(dp) + Sgn(dm) + Sgn(ep) + Sgn(em)
    If check1 < 1 Or check1 > 1 Then N2S = _Trim$(EXP$): Exit Function 'If no scientic notation is found, or if we find more than 1 type, it's not SN!
    Select Case l 'l now tells us where the SN starts at.
        Case Is < dp: l = dp
        Case Is < dm: l = dm
        Case Is < ep: l = ep
        Case Is < em: l = em
    End Select
    l$ = Left$(t$, l - 1) 'The left of the SN
    r$ = Mid$(t$, l + 1): r&& = Val(r$) 'The right of the SN, turned into a workable long
    If InStr(l$, ".") Then 'Location of the decimal, if any
        If r&& > 0 Then
            r&& = r&& - Len(l$) + 2
        Else
            r&& = r&& + 1
        End If
        l$ = Left$(l$, 1) + Mid$(l$, 3)
    End If
    Select Case r&&
        Case 0 'what the heck? We solved it already?
            'l$ = l$
        Case Is < 0
            For i = 1 To -r&&
                l$ = "0" + l$
            Next
            l$ = "." + l$
        Case Else
            For i = 1 To r&&
                l$ = l$ + "0"
            Next
            l$ = l$
    End Select
    N2S$ = sign$ + l$
End Function

Another case solved by string math!

Must always use the result of "STR$()" into the function Steve wrote. I just tried "1e-6" and "N2S$()" returned the same thing. The "D" or "E" to indicate exponent must be uppercase letter. No problem, saying this for people who are used to coding in C, Lua and other languages that prefer lowercase stuff.

PRINT USING and FIX() are a couple of workarounds but for better reliability across a large number spectrum you need to consider using something like my string math routine, ...

This is a scientist's tendency, must be most accurate, try to find the end of PI because there's a beginning...

[bplus]

In IDE, I can not set a number to 1e-6, automatically changed to 1E-6.

Lesson: Always use IDE for QB64 ;-))