-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/zenodo uploader #214
Conversation
Changes to be committed: modified: src/routers/uploader_routers/upload_router_zenodo.py new file: src/tests/uploader/zenodo/mock_zenodo.py modified: src/tests/uploader/zenodo/test_dataset_zenodo_uploader.py modified: src/uploader/zenodo_uploader.py
Changes to be committed: upload_router_huggingface.py moved inside uploader_routers folder, and needed changes on the __init__ files and main.py were made. modified: src/main.py modified: src/routers/__init__.py new file: src/routers/uploader_router.py modified: src/routers/uploader_routers/__init__.py renamed: src/routers/upload_router_huggingface.py -> src/routers/uploader_routers/upload_router_huggingface.py modified: src/routers/uploader_routers/upload_router_zenodo.py modified: src/tests/uploader/zenodo/test_dataset_zenodo_uploader.py uploader folder renamed to uploaders after having zenodo_uploader file created renamed: src/uploader/__init__.py -> src/uploaders/__init__.py renamed: src/uploader/hugging_face_uploader.py -> src/uploaders/hugging_face_uploader.py renamed: src/uploader/utils.py -> src/uploaders/utils.py renamed: src/uploader/zenodo_uploader.py -> src/uploaders/zenodo_uploader.py
Changes to be committed: Minor changes added to improve swagger documentation. modified: src/routers/uploader_routers/upload_router_zenodo.py More tests added and code was cleaned up. modified: src/tests/uploader/zenodo/mock_zenodo.py modified: src/tests/uploader/zenodo/test_dataset_zenodo_uploader.py Publish option added as a paramenter. modified: src/uploaders/zenodo_uploader.py
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Jean!
Generally it looks good, no real problems. Just a lot of small comments.
Confilcts resolved: Conflicts: .gitignore src/main.py src/routers/uploader_routers/upload_router_huggingface.py
…do uploader endpoint.
…r handling invalid platform name and id.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! Looks good - some new comments added
Conflicts: src/routers/uploader_routers/upload_router_huggingface.py On branch feature/zenodo_uploader Changes to be committed: modified: README.md modified: pyproject.toml modified: src/authentication.py modified: src/database/model/agent/contact.py modified: src/database/model/agent/organisation.py modified: src/database/model/concept/concept.py modified: src/database/model/serializers.py modified: src/main.py modified: src/routers/resource_ai_asset_router.py modified: src/routers/resource_router.py modified: src/routers/search_router.py modified: src/tests/routers/generic/test_authentication.py modified: src/tests/routers/resource_routers/test_router_contact.py modified: src/tests/testutils/default_sqlalchemy.py
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me!
Some comments about the regex, and it's good to go
Conflicts (resolved): src/routers/resource_ai_asset_router.py Changes to be committed: modified: pyproject.toml modified: src/routers/resource_ai_asset_router.py modified: src/tests/routers/ai_asset_routers/test_router_aiassets_retrieve_content.py
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great, looks good!
In summary the endpoint
upload/datasets/{identifier}/zenodo
was created to upload content to zenodo.The main changes include:
uploader
was renamed touploaders
andzenodo_uploader.py
was created inside.routers/uploader_routers
was created and fileupload_router_zenodo.py
was created inside.uploader_router_hugginface.py
file was moved inside and the required modifications were done:a. File
uploader_router.py
was created with The abstract classUploaderRouter
. This was passed as a parent toUploadRouterZenodo
, as well as toUploadRouterHuggingface
. The latter was modified to inherit the type of the abstract class and avoid type error with therouter_list
in thesrc/routers/uploader_routers/__init__.py
file.b. Minor changes were made to the
main.py
to include the newrouter_list
Because the zenodo workflow is a bit complex, I'll summarise it as follows: