SystemError: <method 'is_done' of '_thread._ThreadHandle' objects> returned a result with an exception set

[kunom]

Bug report

Bug description:

There is an unexpected SystemError which is completely outside of user code. I get it with Python 3.13.0 on Windows, but did never see it with Python 3.12.x:

Exception ignored on threading shutdown:
Traceback (most recent call last):
  File "C:\Program Files\Python313\Lib\threading.py", line 1524, in _shutdown
    if _main_thread._handle.is_done() and _is_main_interpreter():
SystemError: <method 'is_done' of '_thread._ThreadHandle' objects> returned a result with an exception set

Reproduction

The following minimal reproduction script:

import asyncio
import sys

async def run():
    p = await asyncio.create_subprocess_exec(*["ssh", "[email protected]"])
    try:
        return await p.wait()  # leaving out "return" would help
    finally:
        if p.returncode is None:  # leaving out this "if" statement or the complete "finally" block also would help
            p.terminate()

sys.exit(asyncio.run(run()))

Run the it and leave the ssh session with sudo reboot now. You get the following output (note the last 5 lines):

mek@mek-ubuntu:~$ sudo reboot now
[sudo] password for mek:

Broadcast message from root@mek-ubuntu on pts/1 (Tue 2024-10-22 15:55:11 CEST):

The system will reboot now!

mek@mek-ubuntu:~$ Connection to 192.168.37.128 closed by remote host.
Connection to 192.168.37.128 closed.
Exception ignored on threading shutdown:
Traceback (most recent call last):
  File "C:\Program Files\Python313\Lib\threading.py", line 1524, in _shutdown
    if _main_thread._handle.is_done() and _is_main_interpreter():
SystemError: <method 'is_done' of '_thread._ThreadHandle' objects> returned a result with an exception set

There seems to be a strange interaction with async functions, local variables and return values:

• leaving out the "return" keyword at return await p.wait() prevents the warning message
• leaving out the "finally" block or just leave it empty with pass also omits the warning message
• exiting the SSH session with exit also omits the warning message

CPython versions tested on:

3.13

Operating systems tested on:

Windows

Linked PRs

• gh-125842: Fix sys.exit(0xffff_ffff) on Windows #125896

[colesbury]

cc @mpage.

EDIT : I think it's unlikely that this is a bug in _ThreadHandle.is_done. I don't see how it could set an exception. I think it's more likely that the problematic exception was set before the invocation of _ThreadHandle.is_done.

[colesbury]

I can reproduce this on Windows with:

python -c "import asyncio; raise SystemExit(4294967295)"

(Without the import asyncio, the error message is "OverflowError: Python int too large to convert to C long")

I think the problem is in _Py_HandleSystemExit:

• We don't check the conversion PyLong_AsLong for an error return
• On Windows, long is a 32-bit signed integer
• GetExitCodeProcess returns the exit code (via a pointer) as an unsigned 32-bit integer (DWORD).
• PyLong_AsLong fails due to overflow

cpython/Python/pythonrun.c

Lines 603 to 605 in 91ddde4

	if (PyLong_Check(exc)) {
	exitcode = (int)PyLong_AsLong(exc);
	}

[kunom]

So the issue is that sys.exit(0xffff_ffff) fails on Windows, even though Windows uses 32-bit unsigned integer exit codes.

But since the C standard most probably defines int (at least Microsoft does), what to do?

Would it in this specific case be acceptable to reinterpret-cast the unsigned number, so that 0xffff_ffff renders as -1? The same thing seems to be done by the Windows command line interpreter (see my first link).

[colesbury]

Yes, we should interpret 0xffff_ffff internally as -1 instead of failing with an overflow error. We will also need to fix a few places where _Py_HandleSystemExit can set an exception that isn't expected later on by _Py_Finalize.