Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: decimals are not calculated properly. #261

Open
Flimm opened this issue Aug 22, 2024 · 3 comments
Open

Bug: decimals are not calculated properly. #261

Flimm opened this issue Aug 22, 2024 · 3 comments

Comments

@Flimm
Copy link

Flimm commented Aug 22, 2024

It looks like Heynote uses floating-point arithmetic to calculate equations with decimal numbers. Instead, it should probably use a library like decimal.js in order to make calculations with decimal numbers precisely.

To Reproduce
Steps to reproduce the behavior:

  1. Open a maths block in Heynote
  2. Type 0.1 + 0.2, and you will see 0.3 displayed. So far so good.
  3. Click on the result 0.3 to copy it, and then paste it into a text editor. You will see 0.30000000000000004. This result is incorrect.

Expected behavior
I expect decimal numbers to be calculated properly, without floating-point errors. (I don't want the result to be merely rounded up or down, I want true decimal mathematics using something like decimal.js).

Version: 1.8.0

@NoahAndrews
Copy link
Contributor

Math.js is built on top of decimal.js

@heyman
Copy link
Owner

heyman commented Aug 26, 2024

It's Math.js that is powering the Math blocks, and they write about round-off errors here:
https://mathjs.org/docs/datatypes/numbers.html#roundoff-errors

A solution is to limit the precision just below the actual precision of 16 digits in the displayed output

Heynote uses a lower precision for the displayed values in order to look nicer, while the copied result uses the full precision. However, maybe the problem would be mitigated by using a slightly lower precision for the copied result as well?

@Flimm
Copy link
Author

Flimm commented Aug 26, 2024

Math.js uses floating-point arithmetic by default, which is what is causing these imprecise/incorrect results. To get precise and correct results with Math.js, the documentation recommends using Fractions:

Alternatives are to use Fractions which store a number as a numerator and denominator

// Configure the default type of number: 'number' (default), 'BigNumber', or 'Fraction'
math.config({
  number: 'Fraction'
})

// use the expression parser
math.evaluate('0.32 + 0.08') // Fraction, 2/5

I presume that you can use the format method to convert the Fraction number to string in decimal format, like "0.3".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants