Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
need a Scientific Notation to real number converter
#12
The problem you're seeing is the basic limit of PRINT in printing your value to the screen, as shown below:

Code: (Select All)
PRINT .07000000000000001; ", "; 0.011
PRINT USING ".###, .###"; .07000000000000001, 0.011

The solution is, as was suggested earlier by someone else, to simply swap over to PRINT USING to print your values and have it truncate the result at you specify.

PRINT caps out at DOUBLE precision, from what I remember, while _FLOAT can store values much smaller/larger than that.
Reply


Messages In This Thread
RE: need a Scientific Notation to real number converter - by SMcNeill - 10-12-2023, 02:28 PM



Users browsing this thread: 7 Guest(s)