-
Notifications
You must be signed in to change notification settings - Fork 59
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Transition from using setup.py to pyproject.toml to specify project metadata #269
Merged
Merged
Changes from 12 commits
Commits
Show all changes
16 commits
Select commit
Hold shift + click to select a range
c499b04
define pyproject.toml
R-Palazzo 3ccfce1
update setup.cfg
R-Palazzo 0684101
update makefile
R-Palazzo 3a38772
remove setup.py
R-Palazzo 5919e05
minimum version + lint
R-Palazzo 0d3ab6e
docker
R-Palazzo 7d775e5
update task.py
R-Palazzo 2e7cfb7
formatting
R-Palazzo e89c353
single quote
R-Palazzo 7fac59d
address comments in pyproject.toml
R-Palazzo 3ef342a
add toml dependency
R-Palazzo 1c527e4
update _get_minimum_versions
R-Palazzo a1fd995
pytest
R-Palazzo e8cf5f7
blank lines
R-Palazzo 76d2115
use tomli
R-Palazzo 03c2d87
tomli 2
R-Palazzo File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,17 +1,29 @@ | ||
#!/usr/bin/env python | ||
# -*- coding: utf-8 -*- | ||
|
||
"""The setup script.""" | ||
|
||
from setuptools import find_packages, setup | ||
|
||
with open('README.md', encoding='utf-8') as readme_file: | ||
readme = readme_file.read() | ||
|
||
with open('HISTORY.md', encoding='utf-8') as history_file: | ||
history = history_file.read() | ||
|
||
install_requires = [ | ||
[build-system] | ||
requires = ['setuptools', 'wheel'] | ||
build-backend = 'setuptools.build_meta' | ||
|
||
[project] | ||
name = 'sdgym' | ||
description = 'Benchmark tabular synthetic data generators using a variety of datasets' | ||
authors = [{ name = 'DataCebo, Inc.', email = '[email protected]' }] | ||
classifiers = [ | ||
'Development Status :: 2 - Pre-Alpha', | ||
gsheni marked this conversation as resolved.
Show resolved
Hide resolved
|
||
'Intended Audience :: Developers', | ||
'License :: Free for non-commercial use', | ||
'Natural Language :: English', | ||
'Programming Language :: Python :: 3', | ||
'Programming Language :: Python :: 3.8', | ||
'Programming Language :: Python :: 3.9', | ||
'Programming Language :: Python :: 3.10', | ||
'Programming Language :: Python :: 3.11', | ||
'Topic :: Scientific/Engineering :: Artificial Intelligence', | ||
] | ||
keywords = ['machine learning', 'synthetic data generation', 'benchmark', 'generative models'] | ||
version = '0.7.1.dev0' | ||
license = { text = 'BSL-1.1' } | ||
requires-python = '>=3.8,<3.12' | ||
readme = 'README.md' | ||
dependencies = [ | ||
'appdirs>=1.3,<2', | ||
'boto3>=1.15.0,<2', | ||
'botocore>=1.18,<2', | ||
|
@@ -38,24 +50,29 @@ | |
'sdv>=1.3.0,<2', | ||
] | ||
|
||
[project.urls] | ||
"Source Code"= "https://github.com/sdv-dev/SDGym/" | ||
"Issue Tracker" = "https://github.com/sdv-dev/SDGym/issues" | ||
"Twitter" = "https://twitter.com/sdv_dev" | ||
"Chat" = "https://bit.ly/sdv-slack-invite" | ||
|
||
dask_requires = [ | ||
'dask', | ||
'distributed', | ||
] | ||
[project.entry-points] | ||
sdgym = { main = 'sdgym.cli.__main__:main' } | ||
|
||
setup_requires = [ | ||
'pytest-runner>=2.11.1', | ||
] | ||
[project.optional-dependencies] | ||
dask = ['dask', 'distributed'] | ||
|
||
tests_require = [ | ||
test = [ | ||
'pytest>=3.4.2', | ||
'pytest-cov>=2.6.0', | ||
'jupyter>=1.0.0,<2', | ||
'rundoc>=0.4.3,<0.5', | ||
'toml>=0.10.2,<1', | ||
] | ||
|
||
development_requires = [ | ||
dev = [ | ||
'sdgym[dask, test]', | ||
|
||
# general | ||
'bumpversion>=0.5.3,<0.6', | ||
'pip>=9.0.1', | ||
|
@@ -102,48 +119,24 @@ | |
'invoke', | ||
] | ||
|
||
setup( | ||
author='DataCebo, Inc.', | ||
author_email='[email protected]', | ||
classifiers=[ | ||
'Development Status :: 2 - Pre-Alpha', | ||
'Intended Audience :: Developers', | ||
'License :: Free for non-commercial use', | ||
'Natural Language :: English', | ||
'Programming Language :: Python :: 3', | ||
'Programming Language :: Python :: 3.8', | ||
'Programming Language :: Python :: 3.9', | ||
'Programming Language :: Python :: 3.10', | ||
'Programming Language :: Python :: 3.11', | ||
'Topic :: Scientific/Engineering :: Artificial Intelligence', | ||
], | ||
description=( | ||
'Benchmark tabular synthetic data generators using a variety of datasets' | ||
), | ||
entry_points={ | ||
'console_scripts': [ | ||
'sdgym=sdgym.cli.__main__:main' | ||
], | ||
}, | ||
extras_require={ | ||
'all': development_requires + tests_require + dask_requires, | ||
'dev': development_requires + tests_require + dask_requires, | ||
'test': tests_require, | ||
'dask': dask_requires, | ||
}, | ||
include_package_data=True, | ||
install_requires=install_requires, | ||
license='BSL-1.1', | ||
long_description=readme + '\n\n' + history, | ||
long_description_content_type='text/markdown', | ||
keywords='machine learning synthetic data generation benchmark generative models', | ||
name='sdgym', | ||
packages=find_packages(include=['sdgym', 'sdgym.*']), | ||
python_requires='>=3.8,<3.12', | ||
setup_requires=setup_requires, | ||
test_suite='tests', | ||
tests_require=tests_require, | ||
url='https://github.com/sdv-dev/SDGym', | ||
version='0.7.1.dev0', | ||
zip_safe=False, | ||
) | ||
all = [ | ||
'sdgym[dask, test, dev]', | ||
] | ||
|
||
[tool.setuptools] | ||
include-package-data = true | ||
|
||
[tool.setuptools.packages.find] | ||
include = ['sdgym', 'sdgym.*'] | ||
namespaces = false | ||
|
||
[tool.isort] | ||
line_length = 99 | ||
lines_between_types = 0 | ||
multi_line_output = 4 | ||
not_skip = ['__init__.py'] | ||
use_parentheses = true | ||
|
||
[tool.pydocstyle] | ||
convention = 'google' | ||
add-ignore = ['D107', 'D407', 'D417'] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -2,14 +2,18 @@ | |||||
import inspect | ||||||
import operator | ||||||
import os | ||||||
import re | ||||||
import pkg_resources | ||||||
import platform | ||||||
import re | ||||||
import shutil | ||||||
import stat | ||||||
import sys | ||||||
from pathlib import Path | ||||||
|
||||||
import pkg_resources | ||||||
import toml | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
from invoke import task | ||||||
from packaging.requirements import Requirement | ||||||
from packaging.version import Version | ||||||
|
||||||
COMPARISONS = { | ||||||
'>=': operator.ge, | ||||||
|
@@ -52,50 +56,44 @@ def readme(c): | |||||
os.chdir(cwd) | ||||||
shutil.rmtree(test_path) | ||||||
|
||||||
|
||||||
def _validate_python_version(line): | ||||||
is_valid = True | ||||||
for python_version_match in re.finditer(r"python_version(<=?|>=?|==)\'(\d\.?)+\'", line): | ||||||
python_version = python_version_match.group(0) | ||||||
comparison = re.search(r'(>=?|<=?|==)', python_version).group(0) | ||||||
version_number = python_version.split(comparison)[-1].replace("'", "") | ||||||
comparison_function = COMPARISONS[comparison] | ||||||
is_valid = is_valid and comparison_function( | ||||||
pkg_resources.parse_version(platform.python_version()), | ||||||
pkg_resources.parse_version(version_number), | ||||||
) | ||||||
|
||||||
return is_valid | ||||||
|
||||||
def _get_minimum_versions(dependencies, python_version): | ||||||
min_versions = {} | ||||||
for dependency in dependencies: | ||||||
if '@' in dependency: | ||||||
name, url = dependency.split(' @') | ||||||
min_versions[name] = f'{name} @ {url}' | ||||||
continue | ||||||
|
||||||
req = Requirement(dependency) | ||||||
if ';' in dependency: | ||||||
marker = req.marker | ||||||
if marker and not marker.evaluate({'python_version': python_version}): | ||||||
continue # Skip this dependency if the marker does not apply to the current Python version | ||||||
|
||||||
if req.name not in min_versions: | ||||||
min_version = next((spec.version for spec in req.specifier if spec.operator in ('>=', '==')), None) | ||||||
if min_version: | ||||||
min_versions[req.name] = f'{req.name}=={min_version}' | ||||||
|
||||||
elif '@' not in min_versions[req.name]: | ||||||
existing_version = Version(min_versions[req.name].split('==')[1]) | ||||||
new_version = next((spec.version for spec in req.specifier if spec.operator in ('>=', '==')), existing_version) | ||||||
if new_version > existing_version: | ||||||
min_versions[req.name] = f'{req.name}=={new_version}' # Change when a valid newer version is found | ||||||
|
||||||
return list(min_versions.values()) | ||||||
|
||||||
@task | ||||||
def install_minimum(c): | ||||||
with open('setup.py', 'r') as setup_py: | ||||||
lines = setup_py.read().splitlines() | ||||||
|
||||||
versions = [] | ||||||
started = False | ||||||
for line in lines: | ||||||
if started: | ||||||
if line == ']': | ||||||
started = False | ||||||
continue | ||||||
|
||||||
line = line.strip() | ||||||
if _validate_python_version(line): | ||||||
requirement = re.match(r'[^>]*', line).group(0) | ||||||
requirement = re.sub(r"""['",]""", '', requirement) | ||||||
version = re.search(r'>=?(\d\.?)+\w*', line).group(0) | ||||||
if version: | ||||||
version = re.sub(r'>=?', '==', version) | ||||||
version = re.sub(r"""['",]""", '', version) | ||||||
requirement += version | ||||||
versions.append(requirement) | ||||||
|
||||||
elif (line.startswith('install_requires = [')): | ||||||
started = True | ||||||
|
||||||
c.run(f'python -m pip install {" ".join(versions)}') | ||||||
with open('pyproject.toml', 'r', encoding='utf-8') as pyproject_file: | ||||||
pyproject_data = toml.load(pyproject_file) | ||||||
|
||||||
dependencies = pyproject_data.get('project', {}).get('dependencies', []) | ||||||
python_version = '.'.join(map(str, sys.version_info[:2])) | ||||||
minimum_versions = _get_minimum_versions(dependencies, python_version) | ||||||
|
||||||
if minimum_versions: | ||||||
c.run(f'python -m pip install {" ".join(minimum_versions)}') | ||||||
|
||||||
|
||||||
@task | ||||||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,37 @@ | ||
from tasks import _get_minimum_versions | ||
|
||
|
||
def test_get_minimum_versions(): | ||
"""Test the ``_get_minimum_versions`` method. | ||
|
||
The method should return the minimum versions of the dependencies for the given python version. | ||
If a library is linked to an URL, the minimum version should be the URL. | ||
""" | ||
# Setup | ||
dependencies = [ | ||
"numpy>=1.20.0,<2;python_version<'3.10'", | ||
"numpy>=1.23.3,<2;python_version>='3.10'", | ||
"pandas>=1.2.0,<2;python_version<'3.10'", | ||
"pandas>=1.3.0,<2;python_version>='3.10'", | ||
'humanfriendly>=8.2,<11', | ||
'pandas @ git+https://github.com/pandas-dev/pandas.git@master#egg=pandas' | ||
] | ||
|
||
# Run | ||
minimum_versions_39 = _get_minimum_versions(dependencies, '3.9') | ||
minimum_versions_310 = _get_minimum_versions(dependencies, '3.10') | ||
|
||
# Assert | ||
expected_versions_39 = [ | ||
'numpy==1.20.0', | ||
'pandas @ git+https://github.com/pandas-dev/pandas.git@master#egg=pandas', | ||
'humanfriendly==8.2', | ||
] | ||
expected_versions_310 = [ | ||
'numpy==1.23.3', | ||
'pandas @ git+https://github.com/pandas-dev/pandas.git@master#egg=pandas', | ||
'humanfriendly==8.2', | ||
] | ||
|
||
assert minimum_versions_39 == expected_versions_39 | ||
assert minimum_versions_310 == expected_versions_310 |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wrong library. It is this one