Skip to content

Commit

Permalink
Deprecate the wget backend
Browse files Browse the repository at this point in the history
It doesn't appear that anyone is using it, since I haven't received a
single bug report. And it's a maintenance burden, e.g. I cannot install
posix-spawn on my M1 on Ruby 3.3.1 anymore, and the tests without
posix-spawn are failing for some reason. So, we remove it from the
docs and recommend one of the three other backends.

This will allow us to potentially use WebMock in the future instead of
spinning up a real HTTP server.
  • Loading branch information
janko committed May 2, 2024
1 parent d7d17ee commit 87c8e88
Show file tree
Hide file tree
Showing 4 changed files with 6 additions and 330 deletions.
51 changes: 2 additions & 49 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Down

Down is a utility tool for streaming, flexible and safe downloading of remote
files. It can use [open-uri] + `Net::HTTP`, [http.rb], [HTTPX], or `wget` as
the backend HTTP library.
files. It can use [open-uri] + `Net::HTTP`, [http.rb] or [HTTPX] as the backend
HTTP library.

## Installation

Expand Down Expand Up @@ -235,7 +235,6 @@ The following backends are available:
* [Down::NetHttp](#downnethttp) (default)
* [Down::Http](#downhttp)
* [Down::Httpx](#downhttpx)
* [Down::Wget](#downwget)

You can use the backend directly:

Expand Down Expand Up @@ -465,52 +464,6 @@ io #=> #<Down::ChunkedIO ...>
It's implemented in much of the same way as `Down::Http`, so be sure to check
its docs for ways to pass additional options.

### Down::Wget (experimental)

The `Down::Wget` backend implements downloads using the `wget` command line
utility.

```rb
gem "down", "~> 5.0"
gem "posix-spawn" # omit if on JRuby
gem "http_parser.rb"
```
```rb
require "down/wget"

tempfile = Down::Wget.download("http://nature.com/forest.jpg")
tempfile #=> #<Tempfile:/var/folders/k7/6zx6dx6x7ys3rv3srh0nyfj00000gn/T/20150925-55456-z7vxqz.jpg>

io = Down::Wget.open("http://nature.com/forest.jpg")
io #=> #<Down::ChunkedIO ...>
```

One major advantage of `wget` is that it automatically resumes downloads that
were interrupted due to network failures, which is very useful when you're
downloading large files.

However, the Wget backend should still be considered experimental, as it wasn't
easy to implement a CLI wrapper that streams output, so it's possible that I've
made mistakes. Let me know how it's working out for you 😉.

#### Additional arguments

You can pass additional arguments to the underlying `wget` commmand via symbols:

```rb
Down::Wget.download("http://nature.com/forest.jpg", :no_proxy, connect_timeout: 3)
Down::Wget.open("http://nature.com/forest.jpg", user: "janko", password: "secret")
```

You can also initialize the backend with default arguments:

```rb
wget = Down::Wget.new(:no_proxy, connect_timeout: 3)

wget.download("http://nature.com/forest.jpg")
wget.open("http://nature.com/forest.jpg")
```

## Development

Tests require that a [httpbin] server is running locally, which you can do via Docker:
Expand Down
11 changes: 2 additions & 9 deletions down.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Gem::Specification.new do |spec|

spec.required_ruby_version = ">= 2.3"

spec.summary = "Robust streaming downloads using Net::HTTP, HTTP.rb or wget."
spec.summary = "Robust streaming downloads using Net::HTTP, http.rb or HTTPX."
spec.homepage = "https://github.com/janko/down"
spec.authors = ["Janko Marohnić"]
spec.email = ["[email protected]"]
Expand All @@ -21,13 +21,6 @@ Gem::Specification.new do |spec|
spec.add_development_dependency "mocha", "~> 1.5"
spec.add_development_dependency "rake"
spec.add_development_dependency "httpx", "~> 0", "< 2.0.0"
# http 5.0 drop support of ruby 2.3 and 2.4. We still support those versions.
if RUBY_VERSION >= "2.5"
spec.add_development_dependency "http", "~> 5.0"
else
spec.add_development_dependency "http", "~> 4.3"
end
spec.add_development_dependency "posix-spawn" unless RUBY_ENGINE == "jruby"
spec.add_development_dependency "http_parser.rb" unless RUBY_ENGINE == "jruby"
spec.add_development_dependency "http", RUBY_VERSION >= "2.5" ? "~> 5.0" : "~> 4.3"
spec.add_development_dependency "warning" if RUBY_VERSION >= "2.4"
end
2 changes: 2 additions & 0 deletions lib/down/wget.rb
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# frozen-string-literal: true

warn "The Down::Wget backend has been deprecated and will be removed in the next major version. Please use the Net::HTTP/open-uri, http.rb or HTTPX backend instead."

begin
require "posix-spawn"
rescue LoadError
Expand Down
272 changes: 0 additions & 272 deletions test/wget_test.rb

This file was deleted.

0 comments on commit 87c8e88

Please sign in to comment.