Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why decimal.NewFromFloat(1000) Exponent is 3, and decimal.NewFromInt(1000) Exponent is 0 #374

Open
kkbblzq opened this issue Jun 25, 2024 · 2 comments

Comments

@kkbblzq
Copy link

kkbblzq commented Jun 25, 2024

code

floatVal := decimal.NewFromFloat(1000)
fmt.Println("Exponent: ", floatVal.Exponent())

intVal := decimal.NewFromInt(1000)
fmt.Println("Exponent: ", intVal.Exponent())

output

Exponent:  3
Exponent:  0
@JohnAD
Copy link

JohnAD commented Dec 1, 2024

This is a philosophical question that the library needs to address at some point.

Examples to how to write "1000" using different precision in floating point math:

Expression Significance Precision
1000 unknown unknown
1E3 1 -3
1000. 4 0
1.000E3 4 0
1000.0 5 1

I use the word "Significance" to mean "number of significant digits". I'm not a PhD mathematician so feel free to correct me.

Exponent = -Precision if the precision is negative

These all have the same scalar value, but different precision. Unfortunately, there is no precision stored in a Float64 per the IEEE 754 spec. This library makes a best guess by looking for zeroes. Is that correct? It's more of an opinion question rather than something that has an exact answer. This library is treating it as 1E3. Or "1000 to the nearest 1000".

To give an exact exponent, you can use NewFromFloatWithExponent instead. But this has limited use when dealing with unknown incoming values.

But, an "Integer" in computer-science-y terms is an EXACT number. So, 1000, as an int is exactly 1000, and all three zeroes are significant. Or, written in mathy terms, it is 1.000E3. IMO, the library is handling ints in the most correct way.

If the number is being passed as a string, there is PR to fix the interpretation: #296

@JohnAD
Copy link

JohnAD commented Dec 9, 2024

Or an even better way to term an Integer: an integer has infinite precision due to it's nature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants