Update docs to reflect download-config command (#106)

docs/archival/run-archival-node-without-nearup.md

@@ -97,55 +97,30 @@ The binary path is `target/release/neard`
 The NEAR node requires a working directory with a couple of configuration files. Generate the initial required working directory by running:
 
 ```bash
-$ ./target/release/neard --home ~/.near init --chain-id testnet --download-genesis --download-config
+$ ./target/release/neard --home ~/.near init --chain-id testnet --download-genesis --download-config archival
 ```
 
 > You can specify trusted boot nodes that you'd like to use by pass in a flag during init: `--boot-nodes ed25519:[email protected]:24567,ed25519:[email protected]:24567,ed25519:[email protected]:24567,ed25519:[email protected]:24567`
 
 > You can skip the `--home` argument if you are fine with the default working directory in `~/.near`. If not, pass your preferred location.
 
 This command will create the required directory structure and will generate `config.json`, `node_key.json`, and `genesis.json` for `testnet` network.
-- `config.json` - Configuration parameters which are responsive for how the node will work.
+- `config.json` - Configuration parameters which are responsive for how the node will work. This file should contain the following fields critical for archival nodes:
+  - `"archive": true` - to save the data for all blocks to storage.
+  - `"tracked_shards": [0]` - to track all shards.
 - `genesis.json` - A file with all the data the network started with at genesis. This contains initial accounts, contracts, access keys, and other records which represents the initial state of the blockchain.
 - `node_key.json` -  A file which contains a public and private key for the node. Also includes an optional `account_id` parameter which is required to run a validator node (not covered in this doc).
 - `data/` -  A folder in which a NEAR node will write it's state.
 
 > **Heads up**
 > The genesis file for `testnet` is big (8GB +) so this command will be running for a while and no progress will be shown.
 
-
-### 4. Replacing the `config.json` {#4-replacing-the-configjson}
-
-From the generated `config.json`, there two parameters to modify:
-- `boot_nodes`: If you had not specify the boot nodes to use during init in Step 3, the generated `config.json` shows an empty array, so we will need to replace it with a full one specifying the boot nodes.
-- `tracked_shards`: In the generated `config.json`, this field is an empty empty. You will have to replace it to `"tracked_shards": [0]`
-
-To replace the `config.json`, run the following commands:
-
-```bash
-$ rm ~/.near/config.json
-$ wget https://s3-us-west-1.amazonaws.com/build.nearprotocol.com/nearcore-deploy/testnet/config.json -P ~/.near/
-```
-
-### Configuration Update {#configuration-update}
-
-The `config.json` should contain the following fields. Currently, NEAR testnet and mainnet have 4 shards. To track all 4 shards, use `"tracked_shards": [0]`. In the future, there will be the possibility to track different or multiple shards.
-
-```
-{
-  ...
-  "archive": true,
-  "tracked_shards": [0],
-  ...
-}
-```
-
 Please make sure that the node is not running while changing the `config.json`.
 
 Once the config has been changed, you can restart the node and the node will start syncing new archival data. In the case where you want the full archival history, you can delete the data dir and start the node from scratch syncing full history or use one of the latest backups containing the data directory snapshot which can be copied under the near home dir (default: ~/.near/data).
 
 
-### 5. Get data backup {#5-get-data-backup}
+### 4. Get data backup {#4-get-data-backup}
 
 The node is ready to be started. However, you must first sync up with the network. This means your node needs to download all the headers and blocks that other nodes in the network already have.
 
@@ -155,7 +130,9 @@ $ LATEST=$(cat latest)
 $ aws s3 --no-sign-request cp --no-sign-request --recursive s3://near-protocol-public/backups/testnet/archive/$LATEST ~/.near/data
 ```
 
-### 6. Run the node {#6-run-the-node}
+In the case where you want the full archival history again (for example, if the database gets into an invalid state), you can delete the data dir and start the node from scratch syncing full history or use one of the latest backups containing the data directory snapshot which can be copied under the near home dir (default: ~/.near/data).
+
+### 5. Run the node {#5-run-the-node}
 To start your node simply run the following command:
 
 ```bash
@@ -213,51 +190,23 @@ The binary path is `target/release/neard`
 The NEAR node requires a working directory with a couple of configuration files. Generate the initial required working directory by running:
 
 ```bash
-$ ./target/release/neard --home ~/.near init --chain-id mainnet --download-genesis --download-config
+$ ./target/release/neard --home ~/.near init --chain-id mainnet --download-genesis --download-config archival
 ```
 
 > You can specify trusted boot nodes that you'd like to use by pass in a flag during init: `--boot-nodes ed25519:[email protected]:24567,ed25519:[email protected]:24567,ed25519:[email protected]:24567,ed25519:[email protected]:24567,ed25519:[email protected]:24567`
 
 > You can skip the `--home` argument if you are fine with the default working directory in `~/.near`. If not, pass your preferred location.
 
 This command will create the required directory structure by generating a `config.json`, `node_key.json`, and downloads a `genesis.json` for `mainnet`.
-- `config.json` - Configuration parameters which are responsive for how the node will work.
+- `config.json` - Configuration parameters which are responsive for how the node will work. This file should contain the following fields critical for archival nodes:
+  - `"archive": true` - to save the data for all blocks to storage.
+  - `"tracked_shards": [0]` - to track all shards.
 - `genesis.json` - A file with all the data the network started with at genesis. This contains initial accounts, contracts, access keys, and other records which represents the initial state of the blockchain.
 - `node_key.json` -  A file which contains a public and private key for the node. Also includes an optional `account_id` parameter which is required to run a validator node (not covered in this doc).
 - `data/` -  A folder in which a NEAR node will write it's state.
 
-### 4. Replacing the `config.json` {#4-replacing-the-configjson-1}
-
-From the generated `config.json`, there two parameters to modify:
-- `boot_nodes`: If you had not specify the boot nodes to use during init in Step 3, the generated `config.json` shows an empty array, so we will need to replace it with a full one specifying the boot nodes.
-- `tracked_shards`: In the generated `config.json`, this field is an empty empty. You will have to replace it to `"tracked_shards": [0]`
 
-To replace the `config.json`, run the following commands:
-
-```bash
-$ rm ~/.near/config.json
-$ wget https://s3-us-west-1.amazonaws.com/build.nearprotocol.com/nearcore-deploy/mainnet/config.json -P ~/.near/
-```
-
-### Configuration Update {#configuration-update-1}
-
-The `config.json` should contain the following fields. Currently, NEAR testnet and mainnet have 4 shards. To track all 4 shards, use `"tracked_shards": [0]`. In the future, there will be the possibility to track different or multiple shards.
-
-```
-{
-  ...
-  "archive": true,
-  "tracked_shards": [0],
-  ...
-}
-```
-
-Please make sure that the node is not running while changing the `config.json`.
-
-Once the config has been changed, you can restart the node and the node will start syncing new archival data. In the case where you want the full archival history, you can delete the data dir and start the node from scratch syncing full history or use one of the latest backups containing the data directory snapshot which can be copied under the near home dir (default: ~/.near/data).
-
-
-### 5. Get data backup {#5-get-data-backup-1}
+### 4. Get data backup {#4-get-data-backup-1}
 
 The node is ready to be started. However, you must first sync up with the network. This means your node needs to download all the headers and blocks that other nodes in the network already have.
 
@@ -267,7 +216,9 @@ $ LATEST=$(cat latest)
 $ aws s3 --no-sign-request cp --no-sign-request --recursive s3://near-protocol-public/backups/mainnet/archive/$LATEST ~/.near/data
 ```
 
-### 6. Run the node {#6-run-the-node-1}
+In the case where you want the full archival history again (for example, if the database gets into an invalid state), you can delete the data dir and start the node from scratch syncing full history or use one of the latest backups containing the data directory snapshot which can be copied under the near home dir (default: ~/.near/data).
+
+### 5. Run the node {#5-run-the-node-1}
 To start your node simply run the following command:
 
 ```bash

docs/rpc/run-rpc-node-without-nearup.md

@@ -52,7 +52,7 @@ When building your NEAR node you will have two branch options to choose from dep
 
 ## `testnet`
 
-### 1. Clone `nearcore` project from GitHub
+### 1. Clone `nearcore` project from GitHub {#1-clone-nearcore-project-from-github}
 
 First, clone the [`nearcore` repository](https://github.com/near/nearcore).
 
@@ -68,7 +68,7 @@ Checkout to the branch you need if not `master` (default). Latest release is rec
 $ git checkout tags/1.25.0 -b mynode
 ```
 
-### 2. Compile `nearcore` binary
+### 2. Compile `nearcore` binary {#2-compile-nearcore-binary}
 
 In the `nearcore` folder run the following commands:
 
@@ -90,20 +90,21 @@ consult the `Makefile`, or just stick with `make release`.
 
 The binary path is `target/release/neard`
 
-### 3. Initialize working directory
+### 3. Initialize working directory {#3-initialize-working-directory}
 
 The NEAR node requires a working directory with a couple of configuration files. Generate the initial required working directory by running:
 
 ```bash
-$ ./target/release/neard --home ~/.near init --chain-id testnet --download-genesis --download-config
+$ ./target/release/neard --home ~/.near init --chain-id testnet --download-genesis --download-config rpc
 ```
 
 > You can specify trusted boot nodes that you'd like to use by pass in a flag during init: `--boot-nodes ed25519:[email protected]:24567,ed25519:[email protected]:24567,ed25519:[email protected]:24567,ed25519:[email protected]:24567`
 
 > You can skip the `--home` argument if you are fine with the default working directory in `~/.near`. If not, pass your preferred location.
 
 This command will create the required directory structure and will generate `config.json`, `node_key.json`, and `genesis.json` for `testnet` network.
-- `config.json` - Configuration parameters which are responsive for how the node will work.
+- `config.json` - Configuration parameters which are responsive for how the node will work. This file should contain the following fields critical for RPC nodes:
+  - `"tracked_shards": [0]` - to track all shards.
 - `genesis.json` - A file with all the data the network started with at genesis. This contains initial accounts, contracts, access keys, and other records which represents the initial state of the blockchain.
 - `node_key.json` -  A file which contains a public and private key for the node. Also includes an optional `account_id` parameter which is required to run a validator node (not covered in this doc).
 - `data/` -  A folder in which a NEAR node will write it's state.
@@ -112,20 +113,7 @@ This command will create the required directory structure and will generate `con
 > The genesis file for `testnet` is big (6GB +) so this command will be running for a while and no progress will be shown.
 
 
-### 4. Replacing the `config.json`
-
-From the generated `config.json`, there two parameters to modify:
-- `boot_nodes`: If you had not specify the boot nodes to use during init in Step 3, the generated `config.json` shows an empty array, so we will need to replace it with a full one specifying the boot nodes.
-- `tracked_shards`: In the generated `config.json`, this field is an empty. You will have to replace it to `"tracked_shards": [0]`
-
-To replace the `config.json`, run the following commands:
-
-```bash
-$ rm ~/.near/config.json
-$ wget https://s3-us-west-1.amazonaws.com/build.nearprotocol.com/nearcore-deploy/testnet/config.json -P ~/.near/
-```
-
-### 5. Get data backup
+### 4. Get data backup {#4-get-data-backup}
 
 The node is ready to be started. However, you must first sync up with the network. This means your node needs to download all the headers and blocks that other nodes in the network already have.
 
@@ -166,7 +154,7 @@ $ rclone copy --no-check-certificate --progress --transfers=6 --checkers=6 \
   near_cf://near-protocol-public/backups/testnet/rpc/${latest:?} ~/.near/data
 ```
 
-### 6. Run the node
+### 5. Run the node {#5-run-the-node}
 To start your node simply run the following command:
 
 ```bash
@@ -178,7 +166,7 @@ That's all. The node is running you can see log outputs in your console. It will
 
 ## `mainnet`
 
-### 1. Clone `nearcore` project from GitHub
+### 1. Clone `nearcore` project from GitHub {#1-clone-nearcore-project-from-github-1}
 
 First, clone the [`nearcore` repository](https://github.com/near/nearcore).
 
@@ -196,7 +184,7 @@ For more information on choosing between `master` and latest release branch [ [c
 $ git checkout tags/1.25.0 -b mynode
 ```
 
-### 2. Compile `nearcore` binary
+### 2. Compile `nearcore` binary {#2-compile-nearcore-binary-1}
 
 In the `nearcore` folder run the following commands:
 
@@ -218,38 +206,26 @@ consult the `Makefile`, or just stick with `make release`.
 
 The binary path is `target/release/neard`
 
-### 3. Initialize working directory
+### 3. Initialize working directory {#3-initialize-working-directory-1}
 
 The NEAR node requires a working directory with a couple of configuration files. Generate the initial required working directory by running:
 
 ```bash
-$ ./target/release/neard --home ~/.near init --chain-id mainnet --download-genesis --download-config
+$ ./target/release/neard --home ~/.near init --chain-id mainnet --download-genesis --download-config rpc
 ```
 
 > You can specify trusted boot nodes that you'd like to use by pass in a flag during init: `--boot-nodes ed25519:[email protected]:24567,ed25519:[email protected]:24567,ed25519:[email protected]:24567,ed25519:[email protected]:24567,ed25519:[email protected]:24567`
 
 > You can skip the `--home` argument if you are fine with the default working directory in `~/.near`. If not, pass your preferred location.
 
 This command will create the required directory structure by generating a `config.json`, `node_key.json`, and downloads a `genesis.json` for `mainnet`.
-- `config.json` - Configuration parameters which are responsive for how the node will work.
+- `config.json` - Configuration parameters which are responsive for how the node will work. This file should contain the following fields critical for RPC nodes:
+  - `"tracked_shards": [0]` - to track all shards.
 - `genesis.json` - A file with all the data the network started with at genesis. This contains initial accounts, contracts, access keys, and other records which represents the initial state of the blockchain.
 - `node_key.json` -  A file which contains a public and private key for the node. Also includes an optional `account_id` parameter which is required to run a validator node (not covered in this doc).
 - `data/` -  A folder in which a NEAR node will write it's state.
 
-### 4. Replacing the `config.json`
-
-From the generated `config.json`, there two parameters to modify:
-- `boot_nodes`: If you had not specify the boot nodes to use during init in Step 3, the generated `config.json` shows an empty array, so we will need to replace it with a full one specifying the boot nodes.
-- `tracked_shards`: In the generated `config.json`, this field is an empty. You will have to replace it to `"tracked_shards": [0]`
-
-To replace the `config.json`, run the following commands:
-
-```bash
-$ rm ~/.near/config.json
-$ wget https://s3-us-west-1.amazonaws.com/build.nearprotocol.com/nearcore-deploy/mainnet/config.json -P ~/.near/
-```
-
-### 5. Get data backup
+### 4. Get data backup {#4-get-data-backup-1}
 
 The node is ready to be started. However, you must first sync up with the network. This means your node needs to download all the headers and blocks that other nodes in the network already have.
 
@@ -290,7 +266,7 @@ $ rclone copy --no-check-certificate --progress --transfers=6 --checkers=6 \
   near_cf://near-protocol-public/backups/mainnet/rpc/${latest:?} ~/.near/data
 ```
 
-### 6. Run the node
+### 6. Run the node {#5-run-the-node}
 To start your node simply run the following command:
 
 ```bash

docs/rpc/state-sync.md

@@ -45,9 +45,11 @@ provides access to the state of every shard every epoch:
 Run the following command to download the reference `config.json` file:
 
 ```shell
-./neard --home /tmp/ init --download-genesis --download-config --chain-id <testnet or mainnet>
+./neard --home /tmp/ init --download-genesis --download-config rpc --chain-id <testnet or mainnet>
 ```
 
+*This command is for an RPC node. If you want to download the reference `config.json` file for a validator or archival node, replace the `--download-config rpc` option with `--download-config validator` or `--download-config archival`, respectively.*
+
 The file will be available at `/tmp/config.json`.
 
 ## Troubleshooting
@@ -73,7 +75,7 @@ Check whether state sync is enabled, and check whether it's configured to get st
 If all seems to be configured fine, then disable state sync (set `"state_sync_enabled": false` in `config.json`) and try again.
 If that doesn't help, then restart from a backup data snapshot.
 
-## Running a Chunk-Only Producer that tracks a single shard
+## Running a validator that tracks a single shard
 
 Enable State Sync as explained above.