Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with downloading et.data.get_data() #742

Open
nkorinek opened this issue Oct 28, 2021 · 2 comments
Open

Issue with downloading et.data.get_data() #742

nkorinek opened this issue Oct 28, 2021 · 2 comments
Labels
bug Something isn't working

Comments

@nkorinek
Copy link
Member

When downloading from a url with et.data.get_data(), there is a bug where the file will not be named correctly occasionally. For example, when downloading from https://ndownloader.figshare.com/files/12767966, we would expect the file name to be boulder-temp-2017.txt, however it currently returns a file with the right data in it, but named 12767966 with no extension. This is due to the code looking for a keyword in the header returned from the request. It can't find the keyword even though the download was successful, so it defaults to the last part of the url. Worth noting, the keyword it's looking for exists if the code uses requests.get instead of requests.head in this one case at least. Not sure if that's consistent. Seems to be that maybe a recent update requests changed the name of the headers returned, but not sure!

@lwasser
Copy link

lwasser commented Oct 28, 2021

confirmed i am also getting this same response and i tested on requests version 2.25.1 and 2.26 on a MAC. Nathan also mentioned that we could ensure this fails a bit more gracefully in the future so it's worth looking into that as well. this could be due to an update to the requests module if it's working on some machines but not others.

@lwasser lwasser added the bug Something isn't working label Oct 28, 2021
@nkorinek
Copy link
Member Author

Tried again today and the download worked....still need better error handling!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants