Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Any math experts know what I'm missing here? [Solved]
#13
Quote:Let's say, and I'll just make up a number for this: 4.9665242234315151119541094114141414143678 and it is supposed to be 5.


The example with the number 4.9665 ... which is supposed to be 5 reminded me that I once had to solve a similar problem myself.
A good 20 years ago I wrote an article in a Java magazine about a financial calculation. During programming I had to find out that the corresponding function in Java 1.1 had a nasty bug. If there was a 9 at the third decimal place, something like this: 4,869... then it was not rounded up, but 4,869 became 4.1. Very evil, and very nasty! I only recognized the error because I had previously programmed the same stuff in C.

OK! In Java there was no way to separate the decimal part like in C with fmod(). The only way was to convert the number to a string, check the decimal part, and then convert everything back to a number. See the screenshot.

You can do the same here too. Pseudocode:
Exponent (n) is 5. If the decimal part is 9 in the first place and greater than 5 in the third place, then n = 5. Why still iterations?


[Image: Zahlen-zu-Strings.jpg]
Reply


Messages In This Thread
RE: Any math experts know what I missing here? - by Kernelpanic - 09-23-2022, 08:19 PM



Users browsing this thread: 1 Guest(s)