Skip to content

Commit

Permalink
chore: move away from dev-dependencies
Browse files Browse the repository at this point in the history
`dev-dependencies` is deprecated from poetry.
This PR changes that.

Another benefit from it is that we should now be able to do:
`python -m pip install concrete-ml[dev]` to install from pypi with all
development dependencies.

I looked into dividing into subgroups but there is no way to cleanly
group sub-groups. One has to specify all dependencies in the subgroup
with `{group = "<group_name>"}` instead of the version, which isn't very
convenient.

closes zama-ai/concrete-ml-internal#2005
closes zama-ai/concrete-ml-internal#2319
closes zama-ai/concrete-ml-internal#2685
closes zama-ai/concrete-ml-internal#2685
  • Loading branch information
fd0r committed Jul 29, 2024
1 parent 3d76ba7 commit 01f5581
Show file tree
Hide file tree
Showing 9 changed files with 435 additions and 450 deletions.
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ setup_env:
if [[ $$(uname) != "Linux" ]] && [[ $$(uname) != "Darwin" ]]; then \
poetry install --only dev; \
else \
poetry install; \
poetry install --with dev; \
fi
echo "Finished installing poetry lock."

Expand All @@ -67,7 +67,7 @@ sync_env:
if [[ $$(uname) != "Linux" ]] && [[ $$(uname) != "Darwin" ]]; then \
poetry install --remove-untracked --only dev; \
else \
poetry install --remove-untracked; \
poetry install --remove-untracked --with dev; \
fi; \
"$(MAKE)" setup_env; \
fi
Expand Down
4 changes: 1 addition & 3 deletions ci/aws_ami_build_component.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ phases:
- virtualenv venv --python=python3.9
- echo "source $(pwd)/venv/bin/activate" >> .bashrc
- source venv/bin/activate
- python -m pip install concrete-ml==${CML_VERSION}
- python -m pip install "concrete-ml[dev]==${CML_VERSION}"

- name: test
steps:
Expand Down Expand Up @@ -76,8 +76,6 @@ phases:
commands:
- apt install -y git-lfs
- git lfs install
- source venv/bin/activate
- python -m pip install pytest==7.1.1 pandas==2.0.3 tensorflow==2.12.0 tf2onnx==1.13.0 torchvision==0.14.1 transformers==4.40.0

# We disable tests for test_deploy file because the instance does not have AWS CLI setup
- name: RunTests
Expand Down
2 changes: 1 addition & 1 deletion deps_licenses/licenses_linux_user.txt.md5
Original file line number Diff line number Diff line change
@@ -1 +1 @@
6d367701c3ef5eff8763f4e994e03681
1cde59339dad23a5ecae8ac1192e872e
2 changes: 1 addition & 1 deletion deps_licenses/licenses_mac_intel_user.txt.md5
Original file line number Diff line number Diff line change
@@ -1 +1 @@
adb925c3b7be3e651975febcf49b6543
1cde59339dad23a5ecae8ac1192e872e
12 changes: 8 additions & 4 deletions deps_licenses/licenses_mac_silicon_user.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
Name, Version, License
Jinja2, 3.1.4, BSD License
MarkupSafe, 2.1.5, BSD License
PyYAML, 6.0.1, MIT License
brevitas, 0.8.0, UNKNOWN
brevitas, 0.10.2, UNKNOWN
certifi, 2024.6.2, Mozilla Public License 2.0 (MPL 2.0)
charset-normalizer, 3.3.2, MIT License
coloredlogs, 15.0.1, MIT License
Expand All @@ -12,7 +14,7 @@ flatbuffers, 24.3.25, Apache Software License
fsspec, 2024.6.0, BSD License
huggingface-hub, 0.23.4, Apache Software License
humanfriendly, 10.0, MIT License
hummingbird-ml, 0.4.8, MIT License
hummingbird-ml, 0.4.11, MIT License
idna, 3.7, BSD License
importlib_resources, 6.4.0, Apache Software License
joblib, 1.4.2, BSD License
Expand Down Expand Up @@ -41,10 +43,12 @@ skorch, 0.11.0, new BSD 3-Clause
sympy, 1.12.1, BSD License
tabulate, 0.8.10, MIT License
threadpoolctl, 3.5.0, BSD License
torch, 1.13.1, BSD License
torch, 2.3.1, BSD License
tqdm, 4.66.4, MIT License; Mozilla Public License 2.0 (MPL 2.0)
typing_extensions, 4.5.0, Python Software Foundation License
typing_extensions, 4.12.2, Python Software Foundation License
tzdata, 2024.1, Apache Software License
unfoldNd, 0.2.2, MIT License
urllib3, 2.2.2, MIT License
xgboost, 1.6.2, Apache Software License
z3-solver, 4.13.0.0, MIT License
zipp, 3.19.2, MIT License
2 changes: 1 addition & 1 deletion deps_licenses/licenses_mac_silicon_user.txt.md5
Original file line number Diff line number Diff line change
@@ -1 +1 @@
adb925c3b7be3e651975febcf49b6543
787b1ddb8ddf395cb9c2fd08b57b99c0
837 changes: 413 additions & 424 deletions poetry.lock

Large diffs are not rendered by default.

10 changes: 6 additions & 4 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -58,18 +58,20 @@ numpy = "1.23.5"
protobuf = "3.20.3"
pandas = "2.0.3"

# Needed to have the latest Concrete Python versions
# Zama sources
[[tool.poetry.source]]
name = "zama-pypi-cpu"
url = "https://pypi.zama.ai/cpu"
priority = "explicit"

[tool.poetry.dev-dependencies]
# Development dependencies
[tool.poetry.group.dev]
optional = true

[tool.poetry.group.dev.dependencies]
isort = "^5.10.1"
black = "^24.3.0"
pylint = "^2.13.0"
# Coverage issue
# FIXME: https://github.com/zama-ai/concrete-ml-internal/issues/2541
pytest = "7.4.1"
pytest-cov = "^4.1.0"
pytest-xdist = "^3.3.1"
Expand Down
12 changes: 2 additions & 10 deletions script/make_utils/pytest_pypi_cml.sh
Original file line number Diff line number Diff line change
Expand Up @@ -47,15 +47,7 @@ source "${PYPI_VENV}/bin/activate"
# Install additional dependencies that are required in order to run our tests but are not included
# in PyPI
# Investigate a better way of managing these dependencies
# FIXME: https://github.com/zama-ai/concrete-ml-internal/issues/2685
python -m pip install --upgrade pip
python -m pip install pytest==7.4.1 pandas==2.0.3 tensorflow==2.12.0 tf2onnx==1.15.0 torchvision==0.14.1 transformers==4.40.0

# Install additional pytest plugins
python -m pip install pytest-xdist==3.3.1
python -m pip install pytest-randomly==3.15.0
python -m pip install pytest-repeat==0.9.1
python -m pip install pytest-subtests==0.11.0

if ${USE_PIP_WHEEL}; then
# Delete the directory where the pypi wheel file will be created (if it already exists)
Expand All @@ -70,9 +62,9 @@ if ${USE_PIP_WHEEL}; then
python -m pip install --extra-index-url https://pypi.zama.ai/cpu "${PYPI_WHEEL}"
else
if [ -z "${VERSION}" ]; then
python -m pip install concrete-ml
python -m pip install concrete-ml[dev]
else
python -m pip install concrete-ml=="${VERSION}"
python -m pip install concrete-ml[dev]=="${VERSION}"
fi
fi

Expand Down

0 comments on commit 01f5581

Please sign in to comment.