Python Debugger stops in first few seconds without output

[GlenDC]

Type: Bug

Hello!
I’m on the following specs:

Version: 1.77.3 (Universal)
Commit: 704ed70d4fd1c6bd6342c436f1ede30d1cff4710
Date: 2023-04-12T09:19:37.325Z
Electron: 19.1.11
Chromium: 102.0.5005.196
Node.js: 16.14.2
V8: 10.2.154.26-electron.0
OS: Darwin arm64 22.4.0
Sandboxed: No

I have my custom setup to debug Scrapy-based scraping software written in Python:

{
            "name": "Scrapy Crawl",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/launch_crawler.py",
            "console": "internalConsole",
            "args": [
                "${input:spiders}",
                "${input:crawl_input}",
                "-a",
                "proxyProvider=${input:proxy_provider}",
                "-s",
                "WRITE_CSV=1"
            ],
            "cwd": "${workspaceFolder}",
            "env": {
                "CRAWLERS_SELECTED_SPIDER": "${file}"
            },
            "subProcess": true
        },

This always worked great for me, until I switched to an M2 mac (coming from an intel one).
Now the debugger just exits immediately in the first 3 seconds, while it always worked before…
The thing is, I am confident the issue is somewhere on my host system’s end, e.g. perhaps something
that went wrong in the assisted migration between my two develoment machines, with some binary or something
still bundled for intel rather than arm64.
However, its’ guess work as I do not see any output. Is there somewhere where I can see WHY the debugger exited?
I was looking through the output panes but couldn’t find something from the debugger? Surely I must be able to find somewhere
the reason why it went wrong, no? (

Extension version: 2023.6.1
VS Code version: Code 1.77.3 (Universal) (704ed70d4fd1c6bd6342c436f1ede30d1cff4710, 2023-04-12T09:19:37.325Z)
OS version: Darwin arm64 22.4.0
Modes:
Sandboxed: No

System Info

Item	Value
CPUs	Apple M2 Pro (12 x 24)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled metal: disabled_off multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_renderer: enabled_on video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: disabled_off
Load (avg)	2, 2, 2
Memory (System)	16.00GB (0.07GB free)
Process Argv
Screen Reader	no
VM	0%

[int19h]

The one difference between Intel and ARM Macs would be that the latter don't get the optimized native-code debugger bits. However, pure Python debugger is generally more stable (but slower), so it is surprising to see it crash if that is what's happening here.

Try setting "logToFile": true and have a look at debugpy*.log files in your extension install folder - this might offer some clues.

[GlenDC]

I+00000.024: /handling #1 request "launch" from Adapter/
             Failed to set up process group
             
             Traceback (most recent call last):
               File "/Users/me/.vscode/extensions/ms-python.python-2023.6.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/launcher/debuggee.py", line 74, in preexec_fn
                 tty = os.open("/dev/tty", os.O_RDWR)
             OSError: [Errno 6] Device not configured: '/dev/tty'
             
             Stack where logged:
               File "/Users/me/.pyenv/versions/3.10.3/lib/python3.10/threading.py", line 966, in _bootstrap
                 self._bootstrap_inner()
               File "/Users/me/.pyenv/versions/3.10.3/lib/python3.10/threading.py", line 1009, in _bootstrap_inner
                 self.run()
               File "/Users/me/.pyenv/versions/3.10.3/lib/python3.10/threading.py", line 946, in run
                 self._target(*self._args, **self._kwargs)
               File "/Users/me/.vscode/extensions/ms-python.python-2023.6.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/common/messaging.py", line 1458, in _run_handlers
                 handler()
               File "/Users/me/.vscode/extensions/ms-python.python-2023.6.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/common/messaging.py", line 701, in _handle
                 result = handler(self)
               File "/Users/me/.vscode/extensions/ms-python.python-2023.6.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/launcher/handlers.py", line 140, in launch_request
                 debuggee.spawn(process_name, cmdline, env, redirect_output)
               File "/Users/me/.vscode/extensions/ms-python.python-2023.6.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/launcher/debuggee.py", line 91, in spawn
                 process = subprocess.Popen(cmdline, env=env, bufsize=0, **kwargs)
               File "/Users/me/.pyenv/versions/3.10.3/lib/python3.10/subprocess.py", line 966, in __init__
                 self._execute_child(args, executable, preexec_fn, close_fds,
               File "/Users/me/.pyenv/versions/3.10.3/lib/python3.10/subprocess.py", line 1775, in _execute_child
                 self.pid = _posixsubprocess.fork_exec(
               File "/Users/me/.vscode/extensions/ms-python.python-2023.6.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/launcher/debuggee.py", line 83, in preexec_fn
                 log.swallow_exception(
               File "/Users/me/.vscode/extensions/ms-python.python-2023.6.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/common/log.py", line 215, in swallow_exception
                 _exception(format_string, *args, **kwargs)

Is the only error-ihsh output I can find, this one I find in debugpy.launcher-xxx.log.

[int19h]

This one indicates running outside of a terminal, but this is expected for "internalConsole", and is not actually an error.

debugpy.pydevd*.log is logged from the debuggee and might be more informative. At the very least, if it suddenly stops logging, it would be interesting to see what happened right before then.

[GlenDC]

I see.

This is the last couple of lines of such a session:

0.00s - File not traced (not in project - force_check_project_scope): /Users/<me>/.pyenv/versions/3.10.3/lib/python3.10/_strptime.py
0.00s - File not traced (not in project - force_check_project_scope): /Users/<me>/.pyenv/versions/3.10.3/lib/python3.10/calendar.py
0.00s - File not traced (not in project - force_check_project_scope): /Users/<me>/.pyenv/versions/3.10.3/lib/python3.10/random.py
0.00s - Patching args: ['gcloud', 'config', 'config-helper', '--format', 'json']
0.00s - Process is not python, returning.
1.02s - Patching args: ['gcloud', 'config', 'config-helper', '--format', 'json']
0.00s - Process is not python, returning.
0.67s - File not traced (not in project - force_check_project_scope): /Users/<me>/.pyenv/versions/3.10.3/envs/crawlers/lib/python3.10/site-packages/grpc/_plugin_wrapping.py
0.19s - sending cmd (http_json) -->    CMD_THREAD_CREATE {"type": "event", "event": "thread", "body": {"reason": "started", "threadId": 3}, "seq": 32, "pydevd_cmd_id": 103}

0.00s - Unable to load helper lib to set tracing to all threads (unsupported python vm).
0.00s - Unable to load helper lib to set tracing to all threads (unsupported python vm).
0.00s - Unable to load helper lib to set tracing to all threads (unsupported python vm).
0.11s - Process ThreadsRequest: {
    "command": "threads",
    "seq": 12,
    "type": "request"
}

0.00s - sending cmd (http_json) -->           CMD_RETURN {"type": "response", "request_seq": 12, "success": true, "command": "threads", "body": {"threads": [{"id": 1, "name": "MainThread"}, {"id": 2, "name": "Thread-6 (refresh_data_sources)"}, {"id": 3, "name": "Thread-7 (_run)"}]}, "seq": 34, "pydevd_cmd_id": 502}

0.08s - sending cmd (http_json) -->      CMD_THREAD_KILL {"type": "event", "event": "thread", "body": {"reason": "exited", "threadId": 3}, "seq": 36, "pydevd_cmd_id": 104}

0.08s - File traced: /Users/<my_library>/Middleware/authentication/client_based_authentication_middleware.py
0.00s - File traced: /Users/<my_library>/Middleware/authentication/authentication_middleware.py

Don't think that really helps? Was thinking Unable to load helper lib to set tracing to all threads (unsupported python vm). but that's all over the logs in that file so not sure that is something unexpected?

I feel the answer is obvious if we know why it's exiting though, should it not just tell that somewhere, why? Even if some kind of stack trace or obscure dump?

[int19h]

If it were exiting cleanly then yes, this should be visible in the logs. The fact that the log is cut abruptly like that generally implies a hard crash, not even on Python level but something like a segfault.

I'm not too familiar with macOS, but it seems that any such crashes should be recorded by the OS itself and show up in ~/Library/Logs/DiagnosticReports - can you see if there's anything there? The crash dumps are supposed to have PIDs in the name, which should match the PID used in the filename of the pydevd log.

[GlenDC]

https://gist.github.com/GlenDC/9469f01b4aa6eafa5e075afcfff2cba6

That's an example. I see EXC_BAD_INSTRUCTION. Where it seems to fail to malloc something... But do not not really understand why it happens yet.

[GlenDC]

Wiped my entire pyenv and virtualenv code.
Also did a scan of all intel software and removed all things I found related to legacy installations of Python.

Still same thing... still crashes at start up and still not any closer to knowing why.

[GlenDC]

Tried upgrading Python to 3.11, and upgrade all dependencies of my project...
Yet still get the asme issue :rip:

[GlenDC]

Running the same script with same args using Python directly (even with -m pdb) works.
And yes it's using the same virtualenv, so pretty odd... Probably something small, but as I am not getting any output...

[GlenDC]

You can choose to close the issue if you want. I can once again debug, but don't ask me how, as I'm not sure. Today if just works all of a sudden just again...

Things that have changed recently (as I didn't try it in some weeks):

• i use vscode developer preview
• we use python 3.11 now + all our deps have upgraded

So yeah, no clue what changed, but it works again.