Skip to content

Commit

Permalink
version v1.0.0
Browse files Browse the repository at this point in the history
  • Loading branch information
johnerikhalse committed May 2, 2023
1 parent 856ec9e commit ca8221d
Show file tree
Hide file tree
Showing 11 changed files with 28 additions and 28 deletions.
4 changes: 2 additions & 2 deletions docs/content/cmd/warc.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
date: 2023-03-06T12:35:33+01:00
date: 2023-05-02T10:06:41+02:00
title: "warc"
slug: warc
url: /cmd/warc/
Expand All @@ -11,7 +11,7 @@ A tool for handling warc files
### Options

```
--config string config file. If not set, /etc/warc/, $HOME/.warc/ and current working dir will be searched for file config.yaml
--config string config file. If not set, /etc/xdg/warc, /home/johnh/.config/warc and the current directory will be searched for a file named 'config.yaml'
-h, --help help for warc
--log-console strings the kind of log output to write to console. Valid values: info, error, summary, progress (default [progress,summary])
--log-file strings the kind of log output to write to file. Valid values: info, error, summary (default [info,error,summary])
Expand Down
10 changes: 5 additions & 5 deletions docs/content/cmd/warc_cat.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
date: 2023-03-06T12:35:33+01:00
date: 2023-05-02T10:06:41+02:00
title: "warc cat"
slug: warc_cat
url: /cmd/warc_cat/
Expand Down Expand Up @@ -29,11 +29,11 @@ warc cat -n4 -P file1.warc.gz | feh -
-h, --help help for cat
--id stringArray filter record ID's. For more than one, repeat flag or comma separated list.
-m, --mime-type strings filter records with given mime-types. For more than one, repeat flag or comma separated list.
-n, --num int print the n'th record. This is applied after records are filtered out by other options (default -1)
-o, --offset int print record at offset bytes (default -1)
-n, --num int print the n'th record (zero based). This is applied after records are filtered out by other options (default -1)
-o, --offset int record offset (default -1)
-P, --payload show payload
-p, --protocol-header show protocol header
-c, --record-count int The maximum number of records to show. Defaults to show all records except if -o or -n option is set, then default is one.
-c, --record-count int The maximum number of records to show Defaults to show all records except if -o or -n option is set, then default is one.
-t, --record-type strings filter record types. For more than one, repeat flag or comma separated list.
Legal values: warcinfo,request,response,metadata,revisit,resource,continuation,conversion
-S, --response-code string filter records with given http response codes. Format is 'from-to' where from is inclusive and to is exclusive.
Expand All @@ -47,7 +47,7 @@ warc cat -n4 -P file1.warc.gz | feh -
### Options inherited from parent commands

```
--config string config file. If not set, /etc/warc/, $HOME/.warc/ and current working dir will be searched for file config.yaml
--config string config file. If not set, /etc/xdg/warc, /home/johnh/.config/warc and the current directory will be searched for a file named 'config.yaml'
--log-console strings the kind of log output to write to console. Valid values: info, error, summary, progress (default [progress,summary])
--log-file strings the kind of log output to write to file. Valid values: info, error, summary (default [info,error,summary])
-L, --log-file-name string a file to write log output. Empty for no log file
Expand Down
4 changes: 2 additions & 2 deletions docs/content/cmd/warc_completion.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
date: 2023-03-06T12:35:33+01:00
date: 2023-05-02T10:06:41+02:00
title: "warc completion"
slug: warc_completion
url: /cmd/warc_completion/
Expand Down Expand Up @@ -63,7 +63,7 @@ warc completion [bash|zsh|fish|powershell]
### Options inherited from parent commands

```
--config string config file. If not set, /etc/warc/, $HOME/.warc/ and current working dir will be searched for file config.yaml
--config string config file. If not set, /etc/xdg/warc, /home/johnh/.config/warc and the current directory will be searched for a file named 'config.yaml'
--log-console strings the kind of log output to write to console. Valid values: info, error, summary, progress (default [progress,summary])
--log-file strings the kind of log output to write to file. Valid values: info, error, summary (default [info,error,summary])
-L, --log-file-name string a file to write log output. Empty for no log file
Expand Down
4 changes: 2 additions & 2 deletions docs/content/cmd/warc_console.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
date: 2023-03-06T12:35:33+01:00
date: 2023-05-02T10:06:41+02:00
title: "warc console"
slug: warc_console
url: /cmd/warc_console/
Expand All @@ -22,7 +22,7 @@ warc console <directory> [flags]
### Options inherited from parent commands

```
--config string config file. If not set, /etc/warc/, $HOME/.warc/ and current working dir will be searched for file config.yaml
--config string config file. If not set, /etc/xdg/warc, /home/johnh/.config/warc and the current directory will be searched for a file named 'config.yaml'
--log-console strings the kind of log output to write to console. Valid values: info, error, summary, progress (default [progress,summary])
--log-file strings the kind of log output to write to file. Valid values: info, error, summary (default [info,error,summary])
-L, --log-file-name string a file to write log output. Empty for no log file
Expand Down
4 changes: 2 additions & 2 deletions docs/content/cmd/warc_convert.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
date: 2023-03-06T12:35:33+01:00
date: 2023-05-02T10:06:41+02:00
title: "warc convert"
slug: warc_convert
url: /cmd/warc_convert/
Expand All @@ -17,7 +17,7 @@ Convert web archives to warc files. Use subcommands for the supported formats
### Options inherited from parent commands

```
--config string config file. If not set, /etc/warc/, $HOME/.warc/ and current working dir will be searched for file config.yaml
--config string config file. If not set, /etc/xdg/warc, /home/johnh/.config/warc and the current directory will be searched for a file named 'config.yaml'
--log-console strings the kind of log output to write to console. Valid values: info, error, summary, progress (default [progress,summary])
--log-file strings the kind of log output to write to file. Valid values: info, error, summary (default [info,error,summary])
-L, --log-file-name string a file to write log output. Empty for no log file
Expand Down
6 changes: 3 additions & 3 deletions docs/content/cmd/warc_convert_arc.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
date: 2023-03-06T12:35:33+01:00
date: 2023-05-02T10:06:41+02:00
title: "warc convert arc"
slug: warc_convert_arc
url: /cmd/warc_convert_arc/
Expand All @@ -20,7 +20,7 @@ warc convert arc <files/dirs> [flags]
-c, --concurrency int number of input files to process simultaneously. The default value is 1.5 x <number of cpu cores> (default 24)
-C, --concurrent-writers int maximum concurrent WARC writers. This is the number of WARC-files simultaneously written to.
A consequence is that at least this many WARC files are created even if there is only one input file. (default 1)
-t, --default-date string fetch date to use for records missing date metadata. Fetchtime is set to 12:00 UTC for the date (default "2023-3-6")
-t, --default-date string fetch date to use for records missing date metadata. Fetchtime is set to 12:00 UTC for the date (default "2023-5-2")
-S, --file-size int The maximum size for WARC files (default 1073741824)
--flush if true, sync WARC file to disk after writing each record
-h, --help help for arc
Expand Down Expand Up @@ -49,7 +49,7 @@ warc convert arc <files/dirs> [flags]
### Options inherited from parent commands

```
--config string config file. If not set, /etc/warc/, $HOME/.warc/ and current working dir will be searched for file config.yaml
--config string config file. If not set, /etc/xdg/warc, /home/johnh/.config/warc and the current directory will be searched for a file named 'config.yaml'
--log-console strings the kind of log output to write to console. Valid values: info, error, summary, progress (default [progress,summary])
--log-file strings the kind of log output to write to file. Valid values: info, error, summary (default [info,error,summary])
-L, --log-file-name string a file to write log output. Empty for no log file
Expand Down
6 changes: 3 additions & 3 deletions docs/content/cmd/warc_convert_nedlib.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
date: 2023-03-06T12:35:33+01:00
date: 2023-05-02T10:06:41+02:00
title: "warc convert nedlib"
slug: warc_convert_nedlib
url: /cmd/warc_convert_nedlib/
Expand All @@ -20,7 +20,7 @@ warc convert nedlib <files/dirs> [flags]
-c, --concurrency int number of input files to process simultaneously. The default value is 1.5 x <number of cpu cores> (default 24)
-C, --concurrent-writers int maximum concurrent WARC writers. This is the number of WARC-files simultaneously written to.
A consequence is that at least this many WARC files are created even if there is only one input file. (default 1)
-t, --default-date string fetch date to use for records missing date metadata. Fetchtime is set to 12:00 UTC for the date (default "2023-3-6")
-t, --default-date string fetch date to use for records missing date metadata. Fetchtime is set to 12:00 UTC for the date (default "2023-5-2")
-S, --file-size int The maximum size for WARC files (default 1073741824)
--flush if true, sync WARC file to disk after writing each record
-h, --help help for nedlib
Expand All @@ -47,7 +47,7 @@ warc convert nedlib <files/dirs> [flags]
### Options inherited from parent commands

```
--config string config file. If not set, /etc/warc/, $HOME/.warc/ and current working dir will be searched for file config.yaml
--config string config file. If not set, /etc/xdg/warc, /home/johnh/.config/warc and the current directory will be searched for a file named 'config.yaml'
--log-console strings the kind of log output to write to console. Valid values: info, error, summary, progress (default [progress,summary])
--log-file strings the kind of log output to write to file. Valid values: info, error, summary (default [info,error,summary])
-L, --log-file-name string a file to write log output. Empty for no log file
Expand Down
6 changes: 3 additions & 3 deletions docs/content/cmd/warc_convert_warc.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
date: 2023-03-06T12:35:33+01:00
date: 2023-05-02T10:06:41+02:00
title: "warc convert warc"
slug: warc_convert_warc
url: /cmd/warc_convert_warc/
Expand All @@ -25,7 +25,7 @@ warc convert warc <files/dirs> [flags]
-c, --concurrency int number of input files to process simultaneously. The default value is 1.5 x <number of cpu cores> (default 24)
-C, --concurrent-writers int maximum concurrent WARC writers. This is the number of WARC-files simultaneously written to.
A consequence is that at least this many WARC files are created even if there is only one input file. (default 1)
-t, --default-date string fetch date to use for records missing date metadata. Fetchtime is set to 12:00 UTC for the date (default "2023-3-6")
-t, --default-date string fetch date to use for records missing date metadata. Fetchtime is set to 12:00 UTC for the date (default "2023-5-2")
-S, --file-size int The maximum size for WARC files (default 1073741824)
--flush if true, sync WARC file to disk after writing each record
-h, --help help for warc
Expand Down Expand Up @@ -56,7 +56,7 @@ warc convert warc <files/dirs> [flags]
### Options inherited from parent commands

```
--config string config file. If not set, /etc/warc/, $HOME/.warc/ and current working dir will be searched for file config.yaml
--config string config file. If not set, /etc/xdg/warc, /home/johnh/.config/warc and the current directory will be searched for a file named 'config.yaml'
--log-console strings the kind of log output to write to console. Valid values: info, error, summary, progress (default [progress,summary])
--log-file strings the kind of log output to write to file. Valid values: info, error, summary (default [info,error,summary])
-L, --log-file-name string a file to write log output. Empty for no log file
Expand Down
4 changes: 2 additions & 2 deletions docs/content/cmd/warc_dedup.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
date: 2023-03-06T12:35:33+01:00
date: 2023-05-02T10:06:41+02:00
title: "warc dedup"
slug: warc_dedup
url: /cmd/warc_dedup/
Expand Down Expand Up @@ -67,7 +67,7 @@ warc dedup [flags]
### Options inherited from parent commands

```
--config string config file. If not set, /etc/warc/, $HOME/.warc/ and current working dir will be searched for file config.yaml
--config string config file. If not set, /etc/xdg/warc, /home/johnh/.config/warc and the current directory will be searched for a file named 'config.yaml'
--log-console strings the kind of log output to write to console. Valid values: info, error, summary, progress (default [progress,summary])
--log-file strings the kind of log output to write to file. Valid values: info, error, summary (default [info,error,summary])
-L, --log-file-name string a file to write log output. Empty for no log file
Expand Down
4 changes: 2 additions & 2 deletions docs/content/cmd/warc_ls.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
date: 2023-03-06T12:35:33+01:00
date: 2023-05-02T10:06:41+02:00
title: "warc ls"
slug: warc_ls
url: /cmd/warc_ls/
Expand Down Expand Up @@ -66,7 +66,7 @@ warc ls <files/dirs> [flags]
### Options inherited from parent commands

```
--config string config file. If not set, /etc/warc/, $HOME/.warc/ and current working dir will be searched for file config.yaml
--config string config file. If not set, /etc/xdg/warc, /home/johnh/.config/warc and the current directory will be searched for a file named 'config.yaml'
--log-console strings the kind of log output to write to console. Valid values: info, error, summary, progress (default [progress,summary])
--log-file strings the kind of log output to write to file. Valid values: info, error, summary (default [info,error,summary])
-L, --log-file-name string a file to write log output. Empty for no log file
Expand Down
4 changes: 2 additions & 2 deletions docs/content/cmd/warc_validate.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
date: 2023-03-06T12:35:33+01:00
date: 2023-05-02T10:06:41+02:00
title: "warc validate"
slug: warc_validate
url: /cmd/warc_validate/
Expand All @@ -25,7 +25,7 @@ warc validate <files/dirs> [flags]
### Options inherited from parent commands

```
--config string config file. If not set, /etc/warc/, $HOME/.warc/ and current working dir will be searched for file config.yaml
--config string config file. If not set, /etc/xdg/warc, /home/johnh/.config/warc and the current directory will be searched for a file named 'config.yaml'
--log-console strings the kind of log output to write to console. Valid values: info, error, summary, progress (default [progress,summary])
--log-file strings the kind of log output to write to file. Valid values: info, error, summary (default [info,error,summary])
-L, --log-file-name string a file to write log output. Empty for no log file
Expand Down

0 comments on commit ca8221d

Please sign in to comment.