Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure pip install -e '.[modin-development]' installs all necessary dependencies #1425

Merged
merged 9 commits into from
Apr 25, 2024
Merged

Ensure pip install -e '.[modin-development]' installs all necessary dependencies #1425

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
bea0338
Ensure pip install -e '.[modin-development]' install all necessary de…
sfc-gh-nkrishna Apr 25, 2024
57a1d31
Add CODEOWNERS
sfc-gh-nkrishna Apr 25, 2024
63d61b2
add precommit.yml to both team CODEOWNERS
sfc-gh-nkrishna Apr 25, 2024
d2d1721
add .github to CODEOWNERS for both teams
sfc-gh-nkrishna Apr 25, 2024
8d0f6f5
add scripts/ to CODEOWNERS for both teams
sfc-gh-nkrishna Apr 25, 2024
3425f4e
update setup.py with development requirements
sfc-gh-nkrishna Apr 25, 2024
ba9fc49
add tox.ini to CODENOWNERS
sfc-gh-nkrishna Apr 25, 2024
031918e
add MODIN_REQUIREMENTS
sfc-gh-nkrishna Apr 25, 2024
ad93580
remove list development
sfc-gh-nkrishna Apr 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/CODEOWNERS
View file
Open in desktop
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,7 @@
/src/snowflake/snowpark/modin/ @snowflakedb/snowpandas
/tests/integ/modin/ @snowflakedb/snowpandas
/tests/unit/modin/ @snowflakedb/snowpandas
/.github/ @snowflakedb/snowpandas @snowflakedb/snowpark-python-api-reviewers
/scripts/ @snowflakedb/snowpandas @snowflakedb/snowpark-python-api-reviewers
setup.py @snowflakedb/snowpandas @snowflakedb/snowpark-python-api-reviewers
sfc-gh-nkrishna marked this conversation as resolved.
Show resolved Hide resolved
tox.ini @snowflakedb/snowpandas @snowflakedb/snowpark-python-api-reviewers
2 changes: 1 addition & 1 deletion .github/workflows/precommit.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -531,7 +531,7 @@ jobs:
- name: Upgrade setuptools and pip
run: python -m pip install -U setuptools pip
- name: Install Snowpark
run: python -m pip install ".[modin-development, development, pandas]"
run: python -m pip install ".[modin-development]"
- name: Install Sphinx
run: python -m pip install sphinx
- name: Build document
Expand Down
32 changes: 18 additions & 14 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,20 @@
PANDAS_REQUIREMENTS = [
f"snowflake-connector-python[pandas]{CONNECTOR_DEPENDENCY_VERSION}",
]
MODIN_REQUIREMENTS = [
*PANDAS_REQUIREMENTS,
f"modin{MODIN_DEPENDENCY_VERSION}",
]
DEVELOPMENT_REQUIREMENTS = [
"pytest<8.0.0", # check SNOW-1022240 for more details on the pin here
"pytest-cov",
"coverage",
"sphinx==5.0.2",
"cachetools", # used in UDF doctest
"pytest-timeout",
"pytest-xdist",
"pre-commit",
]

# read the version
VERSION = ()
Expand Down Expand Up @@ -97,28 +111,18 @@
},
extras_require={
"pandas": PANDAS_REQUIREMENTS,
"modin": [
f"modin{MODIN_DEPENDENCY_VERSION}",
*PANDAS_REQUIREMENTS,
],
"modin": MODIN_REQUIREMENTS,
"secure-local-storage": [
f"snowflake-connector-python[secure-local-storage]{CONNECTOR_DEPENDENCY_VERSION}",
],
"development": [
"pytest<8.0.0", # check SNOW-1022240 for more details on the pin here
"pytest-cov",
"coverage",
"sphinx==5.0.2",
"cachetools", # used in UDF doctest
"pytest-timeout",
"pre-commit",
],
"development": DEVELOPMENT_REQUIREMENTS,
"modin-development": [
*MODIN_REQUIREMENTS,
*DEVELOPMENT_REQUIREMENTS,
"pytest-assume", # Snowpark pandas
"decorator", # Snowpark pandas
"scipy", # Snowpark pandas 3rd party library testing
"statsmodels", # Snowpark pandas 3rd party library testing
f"modin{MODIN_DEPENDENCY_VERSION}",
],
"localtest": [
"pandas",
Expand Down
5 changes: 1 addition & 4 deletions tox.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,10 +104,7 @@ commands = {env:SNOWFLAKE_PYTEST_DAILY_CMD} -vvv -m "integ or unit" {posargs:}
[testenv:modin_extra_without_pandas_extra]
allowlist_externals = bash
description = SNOW-1345421: test that we can use modin by specifying the modin extra but not the pandas extra.
deps =
.[development]
.[modin-development]
.[modin]
deps = .[modin-development]
commands = pytest {env:SNOWFLAKE_PYTEST_VERBOSITY:} {env:SNOWFLAKE_PYTEST_COV_CMD} {posargs:} tests/integ/modin/test_modin_extra_without_pandas_extra.py

[testenv:coverage]
Expand Down
Loading