Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overflow issues: geomean amean #25

Open
Zeda opened this issue Feb 21, 2020 · 0 comments
Open

Overflow issues: geomean amean #25

Zeda opened this issue Feb 21, 2020 · 0 comments
Labels
bug Something isn't working good first issue Good for newcomers

Comments

@Zeda
Copy link
Owner

Zeda commented Feb 21, 2020

The way geometric mean (sqrt(x*y)) and arithmetic mean (a+b)/2 is implemented in this library will erroneously cause overflow with some range of inputs. Neither geometric mean nor arithmetic mean (geomean and amean in this library) should overflow except when one or more arguments is +inf or -inf.

Because these are used to compute the Borchardt-Gauss mean, which is used for natural logarithm and inverse trig and inverse hyperbolics and other routines, all of these will erroneously result in overflow for some range of inputs.

As such, this is an important set of bugs to fix.


Ideas:

For amean, if the exponent of either input is close to the maximum, subtract 1 from both exponents, then add the inputs. Otherwise, add the inputs, then subtract 1 from the result exponent. Note: If we go with the first method for all inputs, then we will have erroneous underflow issues.

For geometric mean, it is a bit more complicated. You could compute it as sqrt(x)*sqrt(y), but that erroneously returns NaN if both inputs are negative. It also costs an extra square root.

I think the easiest method is to wrap the existing geomean code in a routine that calculates the output exponent, then sets input exponents to either 0 or 1 as needed.
Basically, if e1 and e2 are the respective exponents, then first do (e1+e2)>>1 → etemp. Set e1 to the carry from computing etemp (so 0 or 1) and e2 to 0. Now compute the product, then the square root, then add etemp to the exponent.

Another method that I contemplated a few years ago here might be useful for the lower-precision floats, but that'll require some investigation.

@Zeda Zeda added bug Something isn't working good first issue Good for newcomers labels Feb 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

1 participant