Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

script/download_files.py: improve logging #25

Open
benoistlaurent opened this issue Aug 17, 2023 · 0 comments
Open

script/download_files.py: improve logging #25

benoistlaurent opened this issue Aug 17, 2023 · 0 comments

Comments

@benoistlaurent
Copy link

Logging level

Logging level is set to logging.DEBUG which makes the script very verbose (at least on stderr), as the user gets debug messages from urllib3.

Better error highlighting

Errors are, IMHO, quite difficult to spot, for two reasons:

  • they are lost in the infinity of debug/info messages.
  • error messages are not displayed using logging.error which makes the message somewhat harder to spot in the log file (you have to use grep -i!).

Example:

2023-08-17 10:59:59 INFO Exception message: 429 Client Error: TOO MANY REQUESTS for url: https://zenodo.org/api/files/dd1e0822-c0f7-43fe-866c-4bc688750c20/PCPS_KCL_500.gro

A summary message at the end of the download process would be nice, e.g. "Successfully downloaded XX files. YY errors. Total size: ZZ Go"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant