Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
need a Scientific Notation to real number converter
#21
Bplus

Can you give an example of string math.  I thought that was what I was doing by converting
the result from number to string then back to a number.  The problem is that when converting 
back to a number it also converted back to the notation.  This has been killing me, The whole 
thing will be removed once I can hand verify that all the calculations are correct.  Right now, as
it is the values don't match.  I print the values to a plain text file so that I can check everything.
Once I can prove everything is calculating correctly this part will be removed from the the main
code.  I could use the notation in my hand calculations but most likely it would increase the odds 
of  me making mistakes.  Speed at this point is not a issue.   

Thanks R1
Reply
#22
(10-20-2023, 02:35 PM)random1 Wrote: Bplus

Can you give an example of string math.  I thought that was what I was doing by converting
the result from number to string then back to a number.  The problem is that when converting 
back to a number it also converted back to the notation.  This has been killing me, The whole 
thing will be removed once I can hand verify that all the calculations are correct.  Right now, as
it is the values don't match.  I print the values to a plain text file so that I can check everything.
Once I can prove everything is calculating correctly this part will be removed from the the main
code.  I could use the notation in my hand calculations but most likely it would increase the odds 
of  me making mistakes.  Speed at this point is not a issue.   

Thanks R1

https://qb64phoenix.com/forum/showthread.php?tid=787
Reply
#23
(10-20-2023, 03:33 PM)SMcNeill Wrote:
(10-20-2023, 02:35 PM)random1 Wrote: Bplus

Can you give an example of string math.  I thought that was what I was doing by converting
the result from number to string then back to a number.  The problem is that when converting 
back to a number it also converted back to the notation.  This has been killing me, The whole 
thing will be removed once I can hand verify that all the calculations are correct.  Right now, as
it is the values don't match.  I print the values to a plain text file so that I can check everything.
Once I can prove everything is calculating correctly this part will be removed from the the main
code.  I could use the notation in my hand calculations but most likely it would increase the odds 
of  me making mistakes.  Speed at this point is not a issue.   

Thanks R1

https://qb64phoenix.com/forum/showthread.php?tid=787

Yeah ScNeill's is good example, I did 18 chars per calc too, due to his influence.
  724  855  599  923  575  468  400  206  147  564  878  823  652  556 bxor cross forever
Reply
#24
@bplus Much better than doing 1 calc per character.  Smile
Reply
#25
(10-20-2023, 04:52 PM)SMcNeill Wrote: @bplus Much better than doing 1 calc per character.  Smile

Yet bet-cha!

@random1 in string math, you never let the numbers you calc with touch QB64 number types they all remain Strings. Remember strings only have a limit of the size memory you have, so in string math the numbers are not limited to any particular and uniform size. For that freedom you pay a price and it may be dear, how long your calculations may take.
  724  855  599  923  575  468  400  206  147  564  878  823  652  556 bxor cross forever
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Maximum Number of Attachments per Thread Magdha 7 457 01-13-2026, 10:13 AM
Last Post: Magdha
  getting the number of dimensions in an array on the fly? madscijr 7 733 09-16-2025, 12:34 AM
Last Post: madscijr
  Random Number Generator pmackay 14 1,257 07-30-2025, 12:56 PM
Last Post: SMcNeill
  Illegal string-number conversion Herve 7 763 07-07-2025, 09:53 AM
Last Post: a740g
  generating a random number in the full range of that number? (Integer, Long) madscijr 2 651 05-01-2025, 09:11 PM
Last Post: madscijr

Forum Jump:


Users browsing this thread: 2 Guest(s)