Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 unarchiving_utils are now based on 7zip cli #6959

Merged
merged 74 commits into from
Jan 7, 2025

Conversation

GitHK
Copy link
Contributor

@GitHK GitHK commented Dec 12, 2024

What do these changes do?

When a new style dynamic service has to fetch an input port containing a zip archive, it will try to unzip it.
If it's not able to open it (see issue here), the service would fail to start.

Important changes:

  • ♻️ dropped used options from archiving and unarchiving
  • 🎨 created drop-in implementation using 7zip cli for archiving and unarchiving
  • 🐛 7zip alo addresses Deflate64 zip-files not supported by dynamic-sidecar powered services #2843
  • 🐛 if a port contains a zip file it will now be removed after it is extracted (and not left behind as it happened before). This also frees up some space for the user on the disk.

Related issue/s

How to test

Dev-ops checklist

@GitHK GitHK self-assigned this Dec 12, 2024
@GitHK GitHK added the a:dynamic-sidecar dynamic-sidecar service label Dec 12, 2024
@GitHK GitHK added bug buggy, it does not work as expected t:maintenance Some planned maintenance work and removed bug buggy, it does not work as expected labels Dec 12, 2024
Copy link

codecov bot commented Dec 12, 2024

Codecov Report

Attention: Patch coverage is 82.22222% with 40 lines in your changes missing coverage. Please review.

Project coverage is 86.80%. Comparing base (c758028) to head (b14d359).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6959      +/-   ##
==========================================
- Coverage   86.89%   86.80%   -0.10%     
==========================================
  Files        1633     1298     -335     
  Lines       64589    55517    -9072     
  Branches     2041     1159     -882     
==========================================
- Hits        56125    48191    -7934     
+ Misses       8129     7114    -1015     
+ Partials      335      212     -123     
Flag Coverage Δ
integrationtests 64.98% <17.85%> (+0.92%) ⬆️
unittests 84.83% <82.22%> (-0.92%) ⬇️
Components Coverage Δ
api ∅ <ø> (∅)
pkg_aws_library 93.49% <ø> (ø)
pkg_dask_task_models_library ∅ <ø> (∅)
pkg_models_library ∅ <ø> (∅)
pkg_notifications_library ∅ <ø> (∅)
pkg_postgres_database ∅ <ø> (∅)
pkg_service_integration ∅ <ø> (∅)
pkg_service_library 73.59% <91.37%> (+0.36%) ⬆️
pkg_settings_library ∅ <ø> (∅)
pkg_simcore_sdk 85.38% <ø> (ø)
agent 96.82% <ø> (ø)
api_server 90.13% <ø> (ø)
autoscaling 96.09% <ø> (ø)
catalog 90.57% <ø> (ø)
clusters_keeper 99.48% <ø> (ø)
dask_sidecar 91.26% <ø> (ø)
datcore_adapter 93.18% <ø> (ø)
director 76.40% <ø> (ø)
director_v2 91.48% <ø> (+0.07%) ⬆️
dynamic_scheduler 97.19% <ø> (ø)
dynamic_sidecar 89.77% <17.85%> (-0.01%) ⬇️
efs_guardian 90.12% <ø> (ø)
invitations 93.44% <ø> (ø)
osparc_gateway_server ∅ <ø> (∅)
payments 92.66% <ø> (ø)
resource_usage_tracker 89.56% <ø> (ø)
storage 89.54% <ø> (ø)
webclient ∅ <ø> (∅)
webserver 84.07% <ø> (-0.01%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c758028...b14d359. Read the comment docs.

@GitHK GitHK marked this pull request as ready for review December 13, 2024 05:47
Copy link
Member

@odeimaiz odeimaiz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the idea behing this change?

Also I was wondering:

  • the dask-sidecar checks the input file mime type. if it is set to zip, then it does not uncompress

@GitHK GitHK requested a review from sanderegg December 13, 2024 10:15
Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please read my additional comments. I don't undersatnd how you wanna present this error to the user.

@GitHK
Copy link
Contributor Author

GitHK commented Dec 20, 2024

@giancarloromeo @matusdrobuliak66 @sanderegg
I've rewritten the entire logic here. please review it as if it were a new.
Thanks a lot

Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, but please let's look at some of the comments. we can have a quick chat.

@GitHK GitHK requested a review from sanderegg January 6, 2025 13:14
Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

services/dynamic-sidecar/Dockerfile Show resolved Hide resolved
Copy link

sonarqubecloud bot commented Jan 7, 2025

@GitHK GitHK merged commit e8046a8 into ITISFoundation:master Jan 7, 2025
89 of 93 checks passed
@GitHK GitHK deleted the pr-osparc-deflate64-zip branch January 7, 2025 13:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:dynamic-sidecar dynamic-sidecar service a:webserver issue related to the webserver service t:maintenance Some planned maintenance work
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Deflate64 zip-files not supported by dynamic-sidecar powered services
6 participants