WIP: Getting-started

docs/Writerside/kc.tree

@@ -7,7 +7,7 @@
 
     <toc-element topic="Overview.md"/>
     <toc-element topic="Getting-started.md">
-        <toc-element topic="Run-Knight-Crawler.md"/>
     </toc-element>
+    <toc-element topic="External-access.md"/>
     <toc-element topic="Supported-Debrid-services.md"/>
 </instance-profile>

docs/Writerside/topics/External-access.md

@@ -0,0 +1,9 @@
+# External access
+
+
+
+In its initial state Knight Crawler will only work on the machine that you host it on, and will not be accessible on
+your local network or from the internet. This is a Stremio
+limitation ([read Stremio's stance here](https://github.com/Stremio/stremio-features/issues/687)) and is beyond our
+control. We provide a guide
+how to make Knight Crawler accessible from your local network or the internet [here]().

docs/Writerside/topics/Getting-started.md

@@ -1,12 +1,7 @@
 # Getting started
 
-Knight Crawler is provided as an all-in-one solution. This means that you can get started with very little setup.
-
-In its initial state Knight Crawler will only work on the machine that you host it on, and will not be accessible on
-your local network or from the internet. This is a Stremio
-limitation ([read Stremio's stance here](https://github.com/Stremio/stremio-features/issues/687)) and is beyond our
-control. We provide a guide
-how to make Knight Crawler accessible from your local network or the internet [here]().
+Knight Crawler is provided as an all-in-one solution. This means we include all the necessary software you need to get started
+out of the box.
 
 ## Before you start
 
@@ -16,3 +11,171 @@ Make sure that you have:
 - [Docker](https://docs.docker.com/get-docker/) and [Compose](https://docs.docker.com/compose/install/) installed
 - A [GitHub](https://github.com/) account _(optional)_
 
+
+## Download the files
+
+Installing Knight Crawler is as simple as downloading a copy of the [deployment directory](https://github.com/Gabisonfire/knightcrawler/tree/master/deployment/docker).
+
+A basic installation requires only two files:
+- <path>deployment/docker/.env.example</path>
+- <path>deployment/docker/docker-compose.yaml</path>.
+
+
+> Knight Crawler will only be accessible on the machine you run it on, to make it accessible from other machines navigate to [External access](External-access.md).
+>
+{style="note"}
+
+For this guide I will be placing them in a directory on my home drive <path>~/knightcrawler</path>.
+
+Rename the <path>.env.example</path> file to be <path>.env</path>
+
+```
+~/
+└── knightcrawler/
+    ├── .env
+    └── docker-compose.yaml
+```
+
+## Initial configuration
+
+Below are a few recommended configuration changes.
+
+Open the <path>.env</path> file in your favourite editor.
+
+> If you are using an external database, configure it in the <path>.env</path> file. Don't forget to disable the ones
+> included in the <path>docker-compose.yaml</path>.
+
+### Database credentials
+
+It is strongly recommended that you change the credentials for the databases included with Knight Crawler. This is best done
+before running Knight Crawler for the first time. It is much harder to change the passwords once the services have been started
+for the first time.
+
+```Bash
+POSTGRES_PASSWORD=postgres
+...
+MONGODB_PASSWORD=mongo
+...
+RABBITMQ_PASSWORD=guest
+```
+
+Here's a few options on generating a secure password:
+
+```Bash
+# Linux
+tr -cd '[:alnum:]' < /dev/urandom | fold -w 64 | head -n 1
+# Or you could use openssl
+openssl rand -hex 32
+```
+```Python
+# Python
+import secrets
+
+print(secrets.token_hex(32))
+```
+
+### Your time zone
+
+```Bash
+TZ=London/Europe
+```
+
+A list of time zones can be found on [Wikipedia](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones)
+
+### Consumers
+
+```Bash
+JOB_CONCURRENCY=5
+...
+MAX_CONNECTIONS_PER_TORRENT=10
+...
+CONSUMER_REPLICAS=3
+```
+
+These are totally subjective to your machine and network capacity. The above default is pretty minimal and will work on
+most machines.
+
+`JOB_CONCURRENCY` is how many films and tv shows the consumers should process at once. As this affects every consumer
+this will likely cause exponential
+strain on your system. It's probably best to leave this at 5, but you can try experimenting with it if you wish.
+
+`MAX_CONNECTIONS_PER_TORRENT` is how many peers the consumer will attempt to connect to when it is trying to collect
+metadata.
+Increasing this value can speed up processing, but you will eventually reach a point where more connections are being
+made than
+your router can handle. This will then cause a cascading fail where your internet stops working. If you are going to
+increase this value
+then try increasing it by 10 at a time.
+
+> Increasing this value increases the max connections for every parallel job, for every consumer. For example
+> with the default values above this means that Knight Crawler will be on average making `(5 x 3) x 10 = 150`
+> connections at any one time.
+>
+{style="warning"}
+
+`CONSUMER_REPLICAS` is how many consumers should be initially started. This is best kept below 10 as GitHub rate limit
+how fast we can access a list of torrent trackers. You can increase or decrease the number of consumers whilst the service is running by running the command `docker compose up --scale consumer=<number>`. This value is best increased by 5 at a time. Repeat this process until you have reached the desired level of consumers.
+
+### GitHub personal access token
+
+This step is optional but strongly recommended. [Debrid Media Manager](https://debridmediamanager.com/start) is a media library manager
+for Debrid services. When a user of this service chooses to export/share their library publicly it is saved to a public GitHub repository.
+This is, essentially, a repository containing a vast amount of ready to go films and tv shows. Knight Crawler comes with the ability to
+read these exported lists, but it requires a GitHub account to make it work.
+
+Knight Crawler needs a personal access token with read-only access to public repositories. This means we can not access any private
+repositories you have.
+
+1. Navigate to GitHub settings ([GitHub token settings](https://github.com/settings/tokens?type=beta)):
+    - Navigate to `GitHub settings`.
+    - Click on `Developer Settings`.
+    - Select `Personal access tokens`.
+    - Choose `Fine-grained tokens`.
+
+2. Press `Generate new token`.
+
+3. Fill out the form with the following information:
+   ```Generic
+   Token name:
+       KnightCrawler
+   Expiration:
+       90 days
+   Description:
+       <blank>
+   Respository access:
+       (checked) Public Repositories (read-only) 
+   ```
+
+4. Click `Generate token`.
+
+5. Take the new token and add it to the bottom of the <path>.env</path> file:
+    ```Bash
+    # Producer
+    GITHUB_PAT=<YOUR TOKEN HERE>
+    ```
+
+## Start Knight Crawler
+
+To start Knight Crawler use the following command:
+
+```Bash
+docker compose up -d
+```
+
+Then we can follow the logs to watch it start:
+
+```Bash
+docker compose logs -f --since 1m
+```
+
+To stop following the logs press <shortcut>Ctrl+C</shortcut> at any time.
+
+The Knight Crawler configuration page should now be accessible in your web browser at [http://localhost:7000](http://localhost:7000)
+
+## Stop Knight Crawler
+
+Knight Crawler can be stopped with the following command:
+
+```Bash
+docker compose down
+```