Skip to content

Commit

Permalink
Refactor
Browse files Browse the repository at this point in the history
  • Loading branch information
WaybackBot committed Aug 21, 2020
1 parent 327114d commit de38f2b
Show file tree
Hide file tree
Showing 9 changed files with 301 additions and 131 deletions.
102 changes: 69 additions & 33 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,6 @@

`wabarc/wayback` is a tool that supports running as a command-line tool and docker container, purpose to snapshot webpage to time capsules.

## Prerequisites

- Golang
- Telegram bot
- Telegram channel (optional)

## Installation

```sh
Expand All @@ -16,8 +10,8 @@ $ go get -u github.com/wabarc/wayback

## Usage

1. Running the command-line or Docker container.
2. Start a chat with the bot and Send URL.
- Running as CLI command or Docker container
- Running with telegram bot

### Command line

Expand All @@ -27,53 +21,95 @@ A CLI tool for wayback webpages.

Usage:
wayback [flags]
wayback [command]

Available Commands:
help Help about any command
telegram A CLI tool for wayback webpages on Telegram bot.
Examples:
wayback https://www.wikipedia.org
wayback https://www.fsf.org https://www.eff.org
wayback --ia https://www.fsf.org
wayback --ip https://www.fsf.org
wayback --ia --is -d telegram -t your-telegram-bot-token
WAYBACK_SLOT=pinata WAYBACK_APIKEY=YOUR-PINATA-APIKEY \
WAYBACK_SECRET=YOUR-PINATA-SECRET wayback --ip https://www.fsf.org

Flags:
-h, --help help for wayback
--host string IPFS daemon host. (default "127.0.0.1")
--port uint IPFS daemon port. (default 5001)
--tor Saving webpage use tor proxy.
-c, --chatid string Channel ID. default: ""
-d, --daemon string Run as daemon service.
--debug Enable debug mode. default: false
-h, --help help for wayback
--ia Wayback webpages to Internet Archive.
--ip Wayback webpages to IPFS. (default false)
--ipfs-host string IPFS daemon host, do not require, unless enable ipfs. (default "127.0.0.1")
-m, --ipfs-mode string IPFS mode. (default "pinner")
-p, --ipfs-port uint IPFS daemon port. (default 5001)
--is Wayback webpages to Archive Today.
-t, --token string Telegram bot API Token, required.
--tor Snapshot webpage use tor proxy.
-v, --version version for wayback
```

Use "wayback [command] --help" for more information about a command.
#### Examples

Wayback one or more url to *Internet Archive* **and** *archive.today*:

```sh
$ wayback https://www.wikipedia.org

$ wayback telegram -t YOUR-BOT-TOKEN
$ wayback https://www.fsf.org https://www.eff.org
```

Publish message to your Telegram channel at the same time:
Wayback url to *Internet Archive* **or** *archive.today* **or** *IPFS*:

```sh
$ wayback telegram
A CLI tool for wayback webpages on Telegram bot.
// Internet Archive
$ wayback --ia https://www.fsf.org

Usage:
wayback telegram [flags]
// archive.today
$ wayback --is https://www.fsf.org

Flags:
-c, --chatid string Channel ID. default: ""
-d, --debug Enable debug mode. default: false
-h, --help help for telegram
-t, --token string Telegram bot API Token, required.
// IPFS
$ wayback --ip https://www.fsf.org
```

For the IPFS, also can use a specific pinner service:

```sh
$ export WAYBACK_SLOT=pinata
$ export WAYBACK_APIKEY=YOUR-PINATA-APIKEY
$ export WAYBACK_SECRET=YOUR-PINATA-SECRET
$ wayback --ip https://www.fsf.org

// or

$ WAYBACK_SLOT=pinata WAYBACK_APIKEY=YOUR-PINATA-APIKEY \
$ WAYBACK_SECRET=YOUR-PINATA-SECRET wayback --ip https://www.fsf.org
```

TIP: [more details](https://github.com/wabarc/ipfs-pinner) about pinner service.

With telegram bot:

$ wayback telegram -t YOUR-BOT-TOKEN -c YOUR-CHANNEL-USERNAME
```sh
$ wayback --ia --is --ip -d telegram -t your-telegram-bot-token
```

Publish message to your Telegram channel at the same time:

```sh
todo
```

Also can run with debug mode:

```sh
$ wayback telegram -t YOUR-BOT-TOKEN -d
$ wayback -d telegram -t YOUR-BOT-TOKEN --debug
```

### Docker/Podman

```sh
$ docker pull wabarc/wayback
$ docker run -d wabarc/wayback telegram -t YOUR-BOT-TOKEN # without telegram channel
$ docker run -d wabarc/wayback telegram -t YOUR-BOT-TOKEN -c YOUR-CHANNEL-USERNAME # with telegram channel
$ docker run -d wabarc/wayback -d telegram -t YOUR-BOT-TOKEN # without telegram channel
$ docker run -d wabarc/wayback -d telegram -t YOUR-BOT-TOKEN -c YOUR-CHANNEL-USERNAME # with telegram channel
```

## TODO
Expand All @@ -92,7 +128,7 @@ $ docker run -d wabarc/wayback telegram -t YOUR-BOT-TOKEN -c YOUR-CHANNEL-USERNA

## Related projects

- duty-machine: <https://github.com/duty-machine/duty-machine>
- [duty-machine](https://github.com/duty-machine/duty-machine)

## License

Expand Down
105 changes: 100 additions & 5 deletions cmd/main.go
Original file line number Diff line number Diff line change
@@ -1,30 +1,125 @@
package main

import (
"fmt"
"os"

"github.com/spf13/cobra"
"github.com/wabarc/wayback"
)

type r = map[string]string

type m struct {
ia r
is r
ip r
}

var (
ia bool
is bool
ip bool

daemon string

host string
port uint
mode string
tor bool

token string
chatid string
debug bool

rootCmd = &cobra.Command{
Use: "wayback",
Short: "A CLI tool for wayback webpages.",
Example: ` wayback https://www.wikipedia.org
wayback https://www.fsf.org https://www.eff.org
wayback --ia https://www.fsf.org
wayback --ip https://www.fsf.org
wayback --ia --is -d telegram -t your-telegram-bot-token
WAYBACK_SLOT=pinata WAYBACK_APIKEY=YOUR-PINATA-APIKEY \
WAYBACK_SECRET=YOUR-PINATA-SECRET wayback --ip https://www.fsf.org`,
Run: func(cmd *cobra.Command, args []string) {
cmd.Help()
// Assign default flags
assign()
handle(cmd, args)
},
Version: "1.0.0",
}
)

var wbrc wayback.Broker

func main() {
rootCmd.Execute()
}

func init() {
rootCmd.PersistentFlags().StringVarP(&host, "host", "", "127.0.0.1", "IPFS daemon host.")
rootCmd.PersistentFlags().UintVarP(&port, "port", "", 5001, "IPFS daemon port.")
rootCmd.PersistentFlags().BoolVarP(&tor, "tor", "", false, "Saving webpage use tor proxy.")
rootCmd.AddCommand(telegramCmd)
rootCmd.Flags().BoolVarP(&ia, "ia", "", false, "Wayback webpages to Internet Archive.")
rootCmd.Flags().BoolVarP(&is, "is", "", false, "Wayback webpages to Archive Today.")
rootCmd.Flags().BoolVarP(&ip, "ip", "", false, "Wayback webpages to IPFS. (default false)")
rootCmd.Flags().StringVarP(&daemon, "daemon", "d", "", "Run as daemon service.")
rootCmd.Flags().StringVarP(&host, "ipfs-host", "", "127.0.0.1", "IPFS daemon host, do not require, unless enable ipfs.")
rootCmd.Flags().UintVarP(&port, "ipfs-port", "p", 5001, "IPFS daemon port.")
rootCmd.Flags().StringVarP(&mode, "ipfs-mode", "m", "pinner", "IPFS mode.")
rootCmd.Flags().BoolVarP(&tor, "tor", "", false, "Snapshot webpage use tor proxy.")

rootCmd.Flags().StringVarP(&token, "token", "t", "", "Telegram bot API Token, required.")
rootCmd.Flags().StringVarP(&chatid, "chatid", "c", "", "Channel ID. default: \"\"")
rootCmd.Flags().BoolVarP(&debug, "debug", "", false, "Enable debug mode. default: false")
}

func assign() {
if !ia && !is && !ip {
ia, is = true, true
}
}

func output(tit string, args r) {
fmt.Printf("[%s]\n", tit)
for ori, dst := range args {
fmt.Printf("%s => %s", ori, dst)
}
fmt.Print("\n")
}

func handle(cmd *cobra.Command, args []string) {
switch daemon {
case "telegram":
telegram()
default:
if len(args) == 0 {
cmd.Help()
os.Exit(0)
}

r := &m{}
if ia {
r.ia = wbia(cmd, args)
output("Internet Archive", r.ia)
}
if is {
r.is = wbis(cmd, args)
output("Archive Today", r.is)
}
if ip {
r.ip = wbip(cmd, args)
output("IPFS", r.ip)
}
}
}

func telegram() {
ipfs := &wayback.IPFSRV{Host: host, Port: port, Mode: mode, UseTor: tor}
handle := map[string]bool{
"ia": ia,
"is": is,
"ip": ip,
}
wbrc := wayback.NewConfig(token, debug, chatid, handle, ipfs)

wbrc.Telegram()
}
37 changes: 0 additions & 37 deletions cmd/telegram.go

This file was deleted.

26 changes: 26 additions & 0 deletions cmd/wayback.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
package main

import (
"github.com/spf13/cobra"
"github.com/wabarc/wayback"
)

func wbia(cmd *cobra.Command, args []string) r {
wbrc = &wayback.Handle{URI: args}

return wbrc.IA()
}

func wbis(cmd *cobra.Command, args []string) r {
wbrc = &wayback.Handle{URI: args}

return wbrc.IS()
}

func wbip(cmd *cobra.Command, args []string) r {
wbrc = &wayback.Handle{
URI: args,
}

return wbrc.WBIPFS()
}
23 changes: 13 additions & 10 deletions config.go
Original file line number Diff line number Diff line change
@@ -1,24 +1,27 @@
package wayback

type IPFS struct {
type IPFSRV struct {
Host string
Port uint
Mode string
UseTor bool
}

type Config struct {
Token string
ChatID string
Debug bool
IPFS *IPFS
Token string
ChatID string
Debug bool
IPFS *IPFSRV
handler map[string]bool
}

func NewConfig(token string, debug bool, chatid string, ipfs *IPFS) *Config {
func NewConfig(token string, debug bool, chatid string, h map[string]bool, ipfs *IPFSRV) *Config {
conf := &Config{
Token: token,
ChatID: chatid,
Debug: debug,
IPFS: ipfs,
Token: token,
ChatID: chatid,
Debug: debug,
IPFS: ipfs,
handler: h,
}

return conf
Expand Down
Loading

0 comments on commit de38f2b

Please sign in to comment.