need a Scientific Notation to real number converter

[random1]

Still giving me problems, I am beginning to think it has something to do with my computer.
Below is a sample of the output file.  I have worked on this for several days and have ran 
out of ideas.    

Thanks all

R1

[SMcNeill]

The problem you're seeing is the basic limit of PRINT in printing your value to the screen, as shown below:

Code: (Select All)

PRINT .07000000000000001; ", "; 0.011

PRINT USING ".###, .###"; .07000000000000001, 0.011

The solution is, as was suggested earlier by someone else, to simply swap over to PRINT USING to print your values and have it truncate the result at you specify.

PRINT caps out at DOUBLE precision, from what I remember, while _FLOAT can store values much smaller/larger than that.

[Jack]

@random1
you are not showing the code of your program, the code I suggested will not print 7.000000000000001D-2 but .070, clearly you are doing something else there
show your code so that we can help

[random1]

@random1
you are not showing the code of your program, the code I suggested will not print 7.000000000000001D-2 but .070, clearly you are doing something else there
show your code so that we can help

The whole program contains almost 30K lines of code.  The code in question, the final step is shown below. 

DCNT, K, KP and FT1 are _fload

K = (KP/FT1)
K$ = StrN$(K, 5, "f")  'RENAMED THE FUNCTION TO StrN$ 
DCNT(L2)=VAL(K$)  ' K$ is stored in DCNT()  as VAL(K$)  'I suspect this is my problem 

The array DCNT() is dimmed as _float and has more than one purpose,  I am thinking this is the problem.
What I need to do is create a string array to hold K$ instead of storing as VAL(K$)       

Mixing data types is a major nemesis for me as I don't fully understand what goes on.  After thinking about
I am sure that saving k$ as VAL(K$) is the problem, I will add a second string array to hold K$ and let you
know if it takes care of the problem.

R1

[random1]

Fixed, even my original code works.  The problem was that saving K$ to val(k$) reverts back to notation.
At least 40 hours wasted.  I would think that after conversion to a string, that VAL(N$) would be a real 
number.

[Jack]

@random1
binary floating point is imprecise especially when converting from/to decimal
if the floating point integer+fraction is a sum of powers of 2 then all is well up to the limit of precision, but anything else is approximate, just like 1/3 can't be represented exactly in decimal .1 can't be represented exactly in binary floating point

[random1]

Jack

Thanks for the help.

Mostly I dim everything double which works well for most projects.  The whole purpose for
writing the data to a file was so that I could go back and check the values by hand.  
It's easy enough to test this  sort of thing outside the main code but I was going to save
some time.

I've been using the StrNum$ function since the late 80's and that's the last thing I expected.  
Turns out I was right, the real issue slid right by me.   

I am pushing 70 and the mind don't work like it once did, maybe it does, it's just I can't 
remember.  LOL  
 
Anyway, thanks again, I was able to pick up on a few things from your post that should 
help in the future.

R1

[Jack]

you are most welcome random1
for business type applications it would be great to have decimal floating point type like the https://www.intel.com/content/www/us/en/...brary.html
I think that it would be far more useful to have a decimal128 type instead of _Float, decimal128 would hold 34 decimal digits

[random1]

Would it be possible to add decimal128 to qb64?  This seems to be what windows calculator uses when 
in scientific mode.  
R1

[bplus]

Would it be possible to add decimal128 to qb64?  This seems to be what windows calculator uses when 
in scientific mode.  
R1

I have 2 words for that: String Math

Then you can have decimal1024 if you want (and don't mind waiting a little longer for calcs) ;-))