-
Notifications
You must be signed in to change notification settings - Fork 200
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bug: decimals are not calculated properly. #261
Comments
Math.js is built on top of decimal.js |
It's Math.js that is powering the Math blocks, and they write about round-off errors here:
Heynote uses a lower precision for the displayed values in order to look nicer, while the copied result uses the full precision. However, maybe the problem would be mitigated by using a slightly lower precision for the copied result as well? |
Math.js uses floating-point arithmetic by default, which is what is causing these imprecise/incorrect results. To get precise and correct results with Math.js, the documentation recommends using Fractions:
I presume that you can use the |
It looks like Heynote uses floating-point arithmetic to calculate equations with decimal numbers. Instead, it should probably use a library like decimal.js in order to make calculations with decimal numbers precisely.
To Reproduce
Steps to reproduce the behavior:
0.1 + 0.2
, and you will see0.3
displayed. So far so good.0.3
to copy it, and then paste it into a text editor. You will see0.30000000000000004
. This result is incorrect.Expected behavior
I expect decimal numbers to be calculated properly, without floating-point errors. (I don't want the result to be merely rounded up or down, I want true decimal mathematics using something like decimal.js).
Version: 1.8.0
The text was updated successfully, but these errors were encountered: