Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[14.0][IMP] datev_export_xml: Improve handling and download of bigger files #158

Merged
merged 7 commits into from
Jun 28, 2024

Conversation

fkantelberg
Copy link
Member

Currently when big files are handled the server might respond with 500 MemoryError when opening the list or form view of the datev xml exports. The worker processes are hitting the RAM limits and will be killed.

@fkantelberg
Copy link
Member Author

@tv-openbig Can you have a look at this PR? Usually it happens with 1GB files which could be hard to generate in the runboat.

The PRs #159 #160

@fkantelberg
Copy link
Member Author

@tv-openbig
I developed the packaging for 14.0. I still need to forward port and test it in 15.0 and 16.0 (don't merge those PRs).

The migration should automatically convert from single file to multiple files per datev export. There is also an indicator in the top right now (exported/total) and the cron only exports invoices not yet exported (e.g. you can delete a specific package to regenerate it without setting everything back to draft). Setting the export to draft will remove all packages. I hope this workflow fits.

@fkantelberg
Copy link
Member Author

@tv-openbig Also was faster than expected. I also finished the changes in 15.0 and 16.0.

@tv-openbig
Copy link
Contributor

@fkantelberg Do you know why the runboat build failed ?

@fkantelberg
Copy link
Member Author

No clue. was running before but got broken over the last weeks

@tv-openbig
Copy link
Contributor

@fkantelberg

The splitting into two .zip archives does not work as expected.

My use case on the linked runbot:
1.) i configured a limit of 20MB
Bildschirmfoto 2024-06-27 um 16 17 10
2.) I uploaded 11 invoices with each invoice one attachment of file size 2.5 MB
Bildschirmfoto 2024-06-27 um 16 14 34
3.) I completed the created supplier bills and created a export of supplier bills
Bildschirmfoto 2024-06-27 um 16 15 24
4.) Here is the list view of the invoices included in the test:
Bildschirmfoto 2024-06-27 um 16 13 12
5.) Everything works fine so far, however i would expect due to my limitation a second .zip archive:
Bildschirmfoto 2024-06-27 um 16 18 58

Bildschirmfoto 2024-06-27 um 16 16 13

@fkantelberg
Copy link
Member Author

@tv-openbig The file is slightly over 20mb and you have compression at work here which might fit. Please create a few more invoices.

@tv-openbig
Copy link
Contributor

@fkantelberg Now it works. Thanks a lot.
Bildschirmfoto 2024-06-28 um 16 08 14

@tv-openbig
Copy link
Contributor

/ocabot merge patch

@OCA-git-bot
Copy link
Contributor

This PR looks fantastic, let's merge it!
Prepared branch 14.0-ocabot-merge-pr-158-by-tv-openbig-bump-patch, awaiting test results.

@OCA-git-bot OCA-git-bot merged commit 1f79e1a into OCA:14.0 Jun 28, 2024
2 checks passed
@OCA-git-bot
Copy link
Contributor

Congratulations, your PR was merged at b567637. Thanks a lot for contributing to OCA. ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants