Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rate limit EBI access #8

Open
s-andrews opened this issue Jun 20, 2019 · 0 comments
Open

Rate limit EBI access #8

s-andrews opened this issue Jun 20, 2019 · 0 comments

Comments

@s-andrews
Copy link

I had a problem trying to retrieve all of the files from SRP190015, of which there are around 900!

It actually works to get the ids, albeit that you have to do it in batches, but it won't set up the download links. It gets variably far through the list, but the "loading" icon always stalls and the list is incomplete.

If I'm reading the code right I think it does a separate query to the EBI warehouse API for each accession, with no rate limiting, so either the repeated requests are causing problems, or they're blocking us for spamming.

I tried to add a delay into the code but just succeeded in breaking it. It might be nice to limit the rate at which these requests go out so EBI won't hate me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant