Provide pluggable/extensible subcommand support

[brettcannon]

This could be used to implement things like py run for .pyz files.

Following the git/cargo approach, py would look for any py--prefixed binary on PATH and then execute it on the behalf of the user (using execve()). It would set the following environment variables:

• PY_LAUNCHER_PATH: the path back to the py binary
• PY_LAUNCHER_VERSION: the version of the Python Launcher (in case that matters)
• PY_LAUNCHER_ARG: the argument to the launcher, if any (e.g. -3.11)

All non-launcher arguments would be passed along to the py-* binary.

The Launcher would gain a way to request interpreter details. From the subcommand, one could do $PY_LAUNCHER_PATH $PY_LAUNCHER_ARG <something> to get the list of known interpreters written to stdout as JSON (like how cargo subcommands work). I don't know if it's worth providing a way to short-circuit the search when an obvious interpreter is found for performance/simplicity of subcommands (e.g. .venv with no version restriction to py), or just always return all results sorted in the order the Launcher would have selected an interpreter? I'm also not sure if this should be exposed as a subcommand (e.g. list --json), a variant of --list (e.g. --list-json), an optional argument to --list (e.g. --list=json:all), or some other UX?

[brettcannon]

Another way to potentially pass in arguments is via JSON and stdin (i.e. pipe).

[encukou]

Does “git-style” include looking in something like git --exec-path in addition to $PATH? On some systems this is used to avoid cluttering $PATH (and tab-completion, etc.) with executables the user isn't meant to run directly.

[brettcannon]

It certainly can. I don't see why some environment variable (or potentially config file) couldn't provide a way to effectively expand PATH just for the Python Launcher.

[brettcannon]

To make it easier to write commands in Python, the Launcher could check if the found command files have a .py or .pyz file extension and execute them with the selected interpreter in those instances. This would make #221 (somewhat) redundant (if the py- prefix requirement is kept then it's not quite the same).

[brettcannon]

I don't know if I knew what a .pyz was before you mentioned it here so maybe I'm an outliner. 😊

Probably not, unfortunately. They were accidentally left out of the What's New when they were first introduced and it seems to have led to the community perpetually not knowing that Python can execute a zip file.

Would you be better off using py-<name>.pyz instead to avoid potential naming collisions, or is that not a concern? (not a question question)

If the file extension wasn't there then I might be more concerned. I also don't expect the .pyz files to have their executable bit set since it isn't necessary (and wouldn't work as an executable without adding a shebang to the file), so I don't expect shells to tab-complete on them. Plus if they are typically accessed via some path specified in some environment variable then they really don't need to be a concern for name clashes.

[brettcannon]

Just had a conversation with someone who argued for just the .pyz idea. 😅

[jefftriplett]

🍿 I think the decision really boils down to do you only want to support writing plugins with Python? I would personally simplify it by searching PATH for py-* files and if they end in pyz, I would strip off the .pyz with tab completion, but my name isn't Brett, Brett. 😉

[brettcannon]

https://discuss.python.org/t/allow-uploading-pyz-zipapp-files-to-pypi/19263/ was a discussion about bringing .pyz files to PyPI. It didn't garner much support, which definitely kills supporting .pyz files only.

I will have to think about whether it makes sense to special-case .pyz files in this instance or simply go the pure git approach for py-* executables.

[brettcannon]

I have developed a py-pip.pyz subcommand which needs the special support for .pyz files (I'll open source the code once I have tested it a bit myself), so that will definitely happen. But I could also see reimplementing the subcommand in Rust for performance, so generic execution will also be supported.

[brettcannon]

Looking at external commands for cargo, it acts like a simple command dispatcher with pointers back to cargo itself.

1. Launching the subcommand cargo {command} ... as cargo-{command} {command} ..., allowing for aliases to provide multiple commands from a single binary.
2. cargo help {command} calls cargo-{command} --help.
3. The $CARGO environment variable is set to point back to cargo itself.
4. The cargo metadata subcommand (and its crate wrapper for easy, programmatic access) provides machine-readable info about the project for commands that need it.

This approach of effectively having subcommands call back into cargo to get interpreter information if they need it solves the concern of environment variables not being able to hold all the environments' information. It could also allow for subcommands to control for whether they want to know about all found interpreters or just the one that the Launcher would have run, and thus operate faster. That would require passing down the Launcher-specific flags, e.g. -3.11 to the subcommand somehow (probably environment variable).

This might mean it makes sense to bake in a py list command with a format control for human vs. machine-readable as well as all interpreter vs. selected/which-style selection of just the interpreter that would be used.

[brettcannon]

Probably should also include the Python Launcher version in case that's needed for some weird backwards-compatibility reason.

[brettcannon]

How gh does external subcommands: https://docs.github.com/en/github-cli/github-cli/creating-github-cli-extensions .