Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Hook for setting Multipart to S3 threshold. #293

Open
DiegoPino opened this issue Jan 19, 2024 · 1 comment
Open

Update Hook for setting Multipart to S3 threshold. #293

DiegoPino opened this issue Jan 19, 2024 · 1 comment
Assignees
Labels
Danger Mr Robinson Things so core to us that need extra care. Please submit automated tests? Digital Preservation Drupal 10 Upgrade Economy (capitalism) enhancement New feature or request
Milestone

Comments

@DiegoPino
Copy link
Member

What?

On the absence of this value, on the latest 1.3.0 branch, all files will be handled via the multipart uploader, which will fail for medium sized (30Mbytes or so)files in MIN.IO is mediating/routing S3 bc of a bug in MIN.IO deprecated S3 routing startup option that sends an invalid header for multipart

This leads to files staying in their original place (or temp) but still being made permanent which means basically yeah, not lost, but also not visible to your IIIF.

This issue will simply set, via an update hook the 5Gbyte setting for you, that way you don't have to go to/admin/config/archipelago/filepersisting and save that form to avoid multipart being the choice always.

I will also open another pull later that will allow, files that ended in a limbo (should not happen but files are important) to be moved a next time/ on ADO re-save.

@DiegoPino DiegoPino self-assigned this Jan 19, 2024
@DiegoPino DiegoPino added enhancement New feature or request Digital Preservation Danger Mr Robinson Things so core to us that need extra care. Please submit automated tests? Drupal 10 Upgrade Economy (capitalism) labels Jan 19, 2024
@DiegoPino DiegoPino added this to the 1.3.0 milestone Jan 19, 2024
@DiegoPino
Copy link
Member Author

The method is here:
https://github.com/esmero/ami/blob/830e6d9c255a4ac9e7e0243945f78cc3f978bcb7/src/Plugin/QueueWorker/IngestADOQueueWorker.php#L1105

Which also reminds me, that the file "identification" is present only for the current user ... wonder if we need to make sure that Hydroponics changes to the user in the queue item? So we have a global state? OR...we don't use the "temp" one and we use the global key storage shared by all users? Suggestions are welcomed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Danger Mr Robinson Things so core to us that need extra care. Please submit automated tests? Digital Preservation Drupal 10 Upgrade Economy (capitalism) enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant