Problem with new dependency checking mechanism in spacy 3.4.2

[b2m]

In spaCy 3.4.2 a new dependency checking mechanism was introduced to the spaCy projects cli.

See #11226 for the addition to 3.4.2 and #11735 for an update.

This dependency check will fail with an exception in 3.4.2 when using some requirements.txt features like pip options or local paths.

See Requirements File Format for a complete list of allowed values in the requirements.txt file.

This is a common use case for Prodigy users (see Installation & Setup).

How to reproduce the behaviour

1. Create minimal requirements.txt

--prefer-binary
spacy==3.4.2

2. Create minimal project.yml

title: "Reproduce Bug"
commands:
  - name: "reproduce"
    script:
      - "echo 'reproducing behaviour'"

3. Run pip install -r requirements.txt
4. Run spacy project run reproduce

Woraround

Set check_requirements: false in the project.yml.

Your Environment

• spaCy version: 3.4.2
• Platform: Linux
• Python version: 3.8.10

[adrianeboyd]

Thanks for the report! This has been fixed in #11735 and the fix will be published soon in v3.4.3. It should skip any lines it can't check with a warning instead of raising an error.

[b2m]

It should skip any lines it can't check with a warning instead of raising an error.

Yes I saw that, however I do not fully agree with the solution.
Following a file specification and then being told by a tool that there are "warnings" feels uncomfortable.

As the structure for the requirements.txt format is quite minimal I would expect a filter filtering only dependencies from the file for the dependency check should be possible.

[adrianeboyd]

I agree, and we'll take a look, although further changes wouldn't make it into v3.4.3. It's frustrating that there's no public API to use pip to do some of the parsing instead.

[adrianeboyd]

We have dropped the requirements checks entirely in weasel due to the deprecation of pkg_resources. If there were a way to implement this correctly/completely (I think it would have to be through pip, which seems unlikely to ever support this), we wouldn't be opposed to adding it back in, but for now there's unfortunately no good way to do it.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.