Python implementation of json.loads() accepts invalid unicode escapes

[nineteendo]

Bug report

Bug description:

While reviewing #125652 and reading the documentation of int(), I realised this condition in json.decoder is insufficient:

cpython/Lib/json/decoder.py

Line 61 in f203d1c

if len(esc) == 4 and esc[1] not in 'xX':

>>> import sys
>>> sys.modules["_json"] = None
>>> import json
>>> json.loads(r'["\u 000", "\u-000", "\u+000", "\u0_00"]')
['\x00', '\x00', '\x00', '\x00']

CPython versions tested on:

3.13

Operating systems tested on:

macOS

Linked PRs

• gh-125660: Python implementation of json.loads() accepts invalid unicode escapes #125683
• [3.13] gh-125660: Reject invalid unicode escapes for Python implementation of JSON decoder (GH-125683) #125694
• [3.12] gh-125660: Reject invalid unicode escapes for Python implementation of JSON decoder (GH-125683) #125695

[nineteendo]

cc @serhiy-storchaka

[nineteendo]

Maybe something like this? Although it might be a better idea to use a stricter function.

esc = s[end:end + 4].strip()
if "_" not in esc and len(esc) == 4 and esc[0] not in "+-" and esc[1] not in "xX":

[serhiy-storchaka]

Either this, or simply use regexp.

[nineteendo]

Let's use a regex, unicode digits aren't allowed either:

>>> int("\uff10", 16)
0

[nineteendo]

Could there be other code that suffers from this problem?

[taleinat]

Could there be other code that suffers from this problem?

Possibly, but that's out of context for this issue. If someone finds such issues they should be reported separately.