Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebDAV storage #98

Merged
merged 3 commits into from
Oct 4, 2024
Merged

WebDAV storage #98

merged 3 commits into from
Oct 4, 2024

Conversation

rgaudin
Copy link
Member

@rgaudin rgaudin commented Oct 3, 2024

backend

  • new WEBDAV_URL_WITH_CREDENTIALS environ
  • new WEBDAV_REQUEST_TIMEOUT_SEC environ (must be long enough for all files to be uploaded in one request)
  • new constants.storage_type: WEBDAV if webdav environ is set, otherwise S3
  • rename all things s3 to storage
  • new abstract StorageInterface which includes Key/Path provider
  • s3 module to implement the StorageInterface
  • new webdav module implementing the StorageInterface
  • New webdav_path property on Project to store the part of URL upload to
  • New API entry point /config to get some constants (public URL)
  • collections.json if present on WebDAV folder is read/parsed and used when adding new files
  • When adding Files from WebDAV, mimetype is read from suffix if not provided (our drives dont set it)
  • .dav endpoint allows add/delete/update of files from WebDAV
  • WebDAV folder is auto created on WebDAV is not existing
  • WebDAV folder is recursively traversed to find Files
  • WebDAV Files discovery auto-filters known garbage (macos files)
  • Removed DOWNLOAD_URL environ: now computed as storage.public_url on S3 and WebDAV
  • Allowing file uploads for projects without expiry (single user mode)
  • <project>.json endpoint to request collection.json generation and upload

frontend

  • Gets public URL from /config call on start
  • Displays and links to public WebDAV URL
  • Project data is replaced on update
  • WebDAV path setter component
  • WebDAV path updater component
  • Fixed New project not changing UI to it
  • Fixed project ordering in sidebar
  • SUCCESS alerts now auto-removing after a delay
  • Error logs as ERROR level instead of generic
  • Files refreshed on WebDAV path set/updated to list discovered

- new WEBDAV_URL_WITH_CREDENTIALS environ
- new WEBDAV_REQUEST_TIMEOUT_SEC environ (must be long enough for all files to be uploaded in one request)
- new `constants.storage_type`: WEBDAV if webdav  environ is set, otherwise S3
- rename all things `s3` to `storage`
- new abstract StorageInterface which includes Key/Path provider
- `s3` module to implement the StorageInterface
- new `webdav` module implementing the StorageInterface
- New `webdav_path` property on Project to store the part of URL upload to

- config entry point
- collections.json remote parsing
- mimetype from suffix
- add/delete/update via .dav endpoint
- auto create remote folder
- display public URL on UI
- recursive listing of files from WeBDAV
- auto filtering of known garbage (macos files)
- UI replaceProject
- UI get /config
- UI conditional display of setter or updater
@rgaudin rgaudin self-assigned this Oct 3, 2024
@rgaudin rgaudin changed the title Webdav storage WebDAV storage Oct 3, 2024
Copy link

codecov bot commented Oct 4, 2024

Codecov Report

Attention: Patch coverage is 29.87342% with 277 lines in your changes missing coverage. Please review.

Project coverage is 61.27%. Comparing base (e92971b) to head (84f3ded).
Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
backend/api/storage/webdav.py 0.00% 102 Missing ⚠️
backend/api/routes/projects.py 22.30% 101 Missing ⚠️
backend/api/storage/s3.py 43.33% 33 Missing and 1 partial ⚠️
backend/api/routes/files.py 22.72% 16 Missing and 1 partial ⚠️
backend/api/storage/__init__.py 68.88% 2 Missing and 12 partials ⚠️
backend/api/routes/archives.py 66.66% 4 Missing and 1 partial ⚠️
backend/api/constants.py 83.33% 1 Missing and 1 partial ⚠️
backend/api/entrypoint.py 50.00% 1 Missing ⚠️
backend/api/routes/utils.py 75.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main      #98       +/-   ##
===========================================
- Coverage   73.24%   61.27%   -11.98%     
===========================================
  Files          18       20        +2     
  Lines         983     1304      +321     
  Branches      120      182       +62     
===========================================
+ Hits          720      799       +79     
- Misses        239      468      +229     
- Partials       24       37       +13     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@rgaudin rgaudin merged commit 2e85fbd into main Oct 4, 2024
3 of 6 checks passed
@rgaudin rgaudin deleted the webdav_storage branch October 4, 2024 11:13
This was linked to issues Oct 4, 2024
This was referenced Oct 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Refresh from WebDAV Upload to WebDAV
1 participant