Bignumber floor issue or misunderstanding of relTol ?

[nycos62]

Hello, I don't know well if it's an issue or if I'm doing something wrong : (math.js version 13.0.3)

math.config({
  number: "BigNumber", // Default type of number:
  // 'number' (default), 'BigNumber', or 'Fraction'
  precision: 509, // Number of significant digits for BigNumbers
  relTol: 1e-60,
  absTol: 1e-63
});

//pi*10^60
math.multiply(math.pi, math.pow(math.bignumber(10),math.bignumber(60))).toFixed()
//'3141592653589793238462643383279502884197169399375105820974944.5923078164062862089986280348253421170679821480865132823066470938446095505822317253594081284811174502841027019385211055596446229489549303819644288109756659334461284756482337867831652712019091456485669234603486104543266482133936072602491412737245870066063155881748815209209628292540917153643678925903600113305305488204665213841469519415116094330572703657595919530921861173819326117931051185480744623799627495673518857527248912279381830119491298336734'

and then because of relTol of 1e-60 when you floor it it send a wrong result :

math.floor(math.multiply(math.pi, math.pow(math.bignumber(10),math.bignumber(60)))).toFixed()
//'3141592653589793238462643383279502884197169399375105820974945' instead of 
//'3141592653589793238462643383279502884197169399375105820974944'

what should I do to get correct number ?
i tried this :

math.config({
  number: "BigNumber", // Default type of number:
  // 'number' (default), 'BigNumber', or 'Fraction'
  precision: 509, // Number of significant digits for BigNumbers
  relTol: 1e-503,
  absTol: 1e-508
});

but it convert then 1e-503 to 0 and then get this error (because relTol = 0 then)

Error: Relative tolerance must be greater than 0

thank you :)

[josdejong]

That is an interesting one. The relTol and absTol values need to be numbers, and a regular Javascript number can "only" hold an exponent up to about 300, so this limits the number of digits we can handle.

So, using a relTol and absTol in the order of 300 will still work:

math.config({
  number: "BigNumber", // Default type of number:
  // 'number' (default), 'BigNumber', or 'Fraction'
  precision: 309, // Number of significant digits for BigNumbers
  relTol: 1e-300,
  absTol: 1e-303
});

math.floor(math.multiply(math.pi, math.pow(math.bignumber(10),math.bignumber(60)))).toFixed()
// "3141592653589793238462643383279502884197169399375105820974944" (as expected)

To address this, I think we can either implement support for relTol and absTol to be a BigNumber, or change them to only note the number of digits (like 300) instead of the value 1e-300.