-
Notifications
You must be signed in to change notification settings - Fork 56
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Operator parseJSONMap fails on JSON objects containing long integer values #2310
Comments
Alright, so this issue seems to be because of CBOR limitations. When executing an operator like StringParseJsonMap, the data is converted first from JSON to CBOR, and then from CBOR to RadonType: JSON -> CBOR -> RadonType A RadonInteger has a limit of
That issue can be fixed by skipping the intermediate CBOR conversion, but then a bigger problem arises: RadonTypes are serialized as CBOR when used in the network protocol. Therefore, as soon as a RadonInteger reaches 2^64, it will be an error to serialize it. For example, this is an error that is seen in the node when a data request uses the IntegerMultiply operator to convert an input of
(the error says tally value but this is the result of the aggregation phase) In that case the data request resolves with 0 commits, because it is not possible to serialize the result. Sheikah doesn't detect that error when using "try data request", so I guess the toolkit will not notice this error as well. So possible solutions for this general integer serialization issue:
I believe all of them are breaking changes. |
On StringParseJsonArray, StringParseJsonMap and StringParseXmlMap, I would suggest to:
|
@tmpolaczyk Shouldn't the node commit something like an "aggregation overflow error" instead? As a matter of fact, there's a specific enum value for this situation defined in the Witnet.sol library. |
Yeah, it should, but this error has never happened before so the current behavior is to not commit anything. Basically here instead of returning an
|
Would it make sense to rephrase the log error message to "Couldn't encode value from < whatever > into < RadonType >: {}" ? |
Yes, that log is wrong. |
Opened #2312 to track the issue of nodes not commiting anything in case of encode error. |
The CBOR RFC seems to indicate that big integers can be supported by representing them as bytes or strings: https://www.rfc-editor.org/rfc/rfc8949.html#name-bignums But not sure if this solution works well for our case, because:
|
Now I'm curious of which integer numbers cannot be formatted as strings 🤔 |
I guess most of them, for example -100000000000000000000 is encoded as
And |
For instance:
{"data": 122637770461000000000}
The text was updated successfully, but these errors were encountered: