Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
decfloat -- again
#26
On my ad/subtract/multiply/divide calculator, I just kept all results as fractions, and displayed the results as a decimal. It works great for simple operations. Very easy to go in one direction, back in the other, and end up with same number you started with. However, I discovered that method is daunting when it comes to dealing with logs, decimal powers and decimal roots. Rounding is the alternative, but I haven't found any definitive source to build a reliable routine around. It's weird, but I suppose some results that look like rounding to 2 would be correct, but how about in a case where 1.99999999999999999902 is the correct answer? So we know we can't just make rounding rules based on assumptions, and more importantly, for a high precision calculator to be accepted, it would have to conform to some apparent "universal" rounding standards, whatever the heck they are. Unfortunately, my last name isn't Casio. Have you found any authoritative source to work from?

Pete
Reply


Messages In This Thread
decfloat -- again - by Jack - 09-13-2022, 08:45 PM
RE: decfloat -- again - by Jack - 09-13-2022, 08:48 PM
RE: decfloat -- again - by Jack - 09-14-2022, 02:52 PM
RE: decfloat -- again - by Pete - 09-14-2022, 04:10 PM
RE: decfloat -- again - by SpriggsySpriggs - 09-14-2022, 04:31 PM
RE: decfloat -- again - by Jack - 09-14-2022, 06:26 PM
RE: decfloat -- again - by SpriggsySpriggs - 09-14-2022, 09:30 PM
RE: decfloat -- again - by Pete - 09-14-2022, 07:57 PM
RE: decfloat -- again - by BSpinoza - 09-15-2022, 03:27 AM
RE: decfloat -- again - by Pete - 09-15-2022, 03:37 AM
RE: decfloat -- again - by SpriggsySpriggs - 09-15-2022, 01:56 PM
RE: decfloat -- again - by Jack - 09-16-2022, 09:03 PM
RE: decfloat -- again - by Pete - 09-16-2022, 10:31 PM
RE: decfloat -- again - by Jack - 09-17-2022, 12:19 AM
RE: decfloat -- again - by Pete - 09-17-2022, 12:40 AM
RE: decfloat -- again - by Jack - 09-19-2022, 01:48 AM
RE: decfloat -- again - by Pete - 09-20-2022, 02:30 AM
RE: decfloat -- again - by SpriggsySpriggs - 09-19-2022, 01:08 PM
RE: decfloat -- again - by Jack - 09-20-2022, 12:18 PM
RE: decfloat -- again - by Pete - 09-20-2022, 07:02 PM
RE: decfloat -- again - by Kernelpanic - 09-20-2022, 09:54 PM
RE: decfloat -- again - by Jack - 10-08-2022, 05:51 PM
RE: decfloat -- again - by Pete - 10-09-2022, 05:09 PM
RE: decfloat -- again - by Jack - 10-09-2022, 07:21 PM
RE: decfloat -- again - by Jack - 10-12-2022, 01:00 AM
RE: decfloat -- again - by Pete - 10-12-2022, 01:33 AM
RE: decfloat -- again - by Jack - 10-12-2022, 01:43 AM
RE: decfloat -- again - by Pete - 10-12-2022, 02:12 AM
RE: decfloat -- again - by Jack - 10-14-2022, 12:04 AM
RE: decfloat -- again - by Pete - 10-14-2022, 02:55 AM
RE: decfloat -- again - by Jack - 10-14-2022, 11:32 AM
RE: decfloat -- again - by Jack - 10-14-2022, 01:41 PM
RE: decfloat -- again - by Pete - 10-14-2022, 06:36 PM
RE: decfloat -- again - by Jack - 10-14-2022, 08:09 PM
RE: decfloat -- again - by Pete - 10-14-2022, 08:20 PM
RE: decfloat -- again - by Jack - 10-14-2022, 08:28 PM
RE: decfloat -- again - by Pete - 10-14-2022, 08:45 PM
RE: decfloat -- again - by Jack - 10-14-2022, 08:56 PM
RE: decfloat -- again - by Pete - 10-14-2022, 09:06 PM
RE: decfloat -- again - by Jack - 10-25-2022, 10:32 PM
RE: decfloat -- again - by Pete - 10-25-2022, 10:56 PM



Users browsing this thread: 2 Guest(s)