Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
need a Scientific Notation to real number converter
#14
(10-12-2023, 02:45 PM)Jack Wrote: @random1
you are not showing the code of your program, the code I suggested will not print 7.000000000000001D-2 but .070, clearly you are doing something else there
show your code so that we can help
The whole program contains almost 30K lines of code.  The code in question, the final step is shown below. 

DCNT, K, KP and FT1 are _fload

K = (KP/FT1)
K$ = StrN$(K, 5, "f")  'RENAMED THE FUNCTION TO StrN$ 
DCNT(L2)=VAL(K$)  ' K$ is stored in DCNT()  as VAL(K$)  'I suspect this is my problem 

The array DCNT() is dimmed as _float and has more than one purpose,  I am thinking this is the problem.
What I need to do is create a string array to hold K$ instead of storing as VAL(K$)       

Mixing data types is a major nemesis for me as I don't fully understand what goes on.  After thinking about
I am sure that saving k$ as VAL(K$) is the problem, I will add a second string array to hold K$ and let you
know if it takes care of the problem.

R1
Reply


Messages In This Thread
RE: need a Scientific Notation to real number converter - by random1 - 10-13-2023, 05:23 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Maximum Number of Attachments per Thread Magdha 7 470 01-13-2026, 10:13 AM
Last Post: Magdha
  getting the number of dimensions in an array on the fly? madscijr 7 745 09-16-2025, 12:34 AM
Last Post: madscijr
  Random Number Generator pmackay 14 1,274 07-30-2025, 12:56 PM
Last Post: SMcNeill
  Illegal string-number conversion Herve 7 774 07-07-2025, 09:53 AM
Last Post: a740g
  generating a random number in the full range of that number? (Integer, Long) madscijr 2 662 05-01-2025, 09:11 PM
Last Post: madscijr

Forum Jump:


Users browsing this thread: 1 Guest(s)