wget

Jump to bottom

Tim L edited this page Jul 19, 2014 · 6 revisions

What we will cover

This page describes some useful wget options. We use http://data.semanticweb.org/dumps/ as an example site to retrieve.

Let's get to it

Options and what they mean is shown here

mkdir -p source && pushd source
   wget --mirror -e robots=off --no-parent $web_directory
popd

If you're just looking for RDF, wget --mirror -e robots=off -A owl,rdf,ttl,nt,n3 --no-parent $web_directory

What is next

Places we use wget:

Clone this wiki locally