Uncaught timeout exception in http.rb

[danshearer]

I got this traceback:

/usr/lib/ruby/2.7.0/net/http.rb:960:in `initialize': execution expired (Net::OpenTimeout)

This was with the wayback_archive 1.4.0 gem, reproducible with latest github code.

I'm not sure if this project is still maintained, but thanks anyway.

Dan

[buren]

Hi @danshearer,

I've been meaning to give this gem some love. I definitely intend to maintain it.

This particular error looks like it might not be an issue with wayback_archiver, it rather looks like the request timed out. Net::HTTP will raise that exception if it takes more than 60 seconds to open a connections (documentation for Net:HTTP for Ruby 2.7).

Some questions to help me debug if there's an issue 👇
How are you invoking wayback_archiver? CLI or with Ruby code and can you paste it here?
Does this happen consistently? Does it work if you try archiving another site?

Thanks!

[danshearer]

Oh it is certainly a timeout issue.

But I was thinking that really this should be caught and presented as an error, and potentially even a back off and retry? It's a different case to being hard rartelimited, and is hopefully transient.

I was thinking that a backtrace was a little brutal, given this context?

[buren]

I see where you come from. I'm not entirely sure that I'd want to try to capture and re-raise each possible exception, however there's definitely a use case for retrying failed requests. I've added that as a possible feature in the v2 roadmap 👉 #68.