From 7192fefde6ca6950940a7a0ff9a98565c231f3c8 Mon Sep 17 00:00:00 2001 From: Konstantin Pandl Date: Sat, 25 May 2024 00:37:48 +0200 Subject: [PATCH 01/19] hw specs and script start command --- README.md | 106 ++++++++++++++++++++++++++++++++++---------- run-client.sh | 40 ----------------- run-core-client.sh | 48 ++++++++++++++++++++ run-outer-client.sh | 48 ++++++++++++++++++++ run-prover.sh | 25 +++++------ run-validator.sh | 57 ++++++++++++++++-------- 6 files changed, 227 insertions(+), 97 deletions(-) delete mode 100755 run-client.sh create mode 100755 run-core-client.sh create mode 100755 run-outer-client.sh diff --git a/README.md b/README.md index 4f90a2aaa8..f1fb130929 100644 --- a/README.md +++ b/README.md @@ -38,37 +38,40 @@ which verifies transactions and stores the encrypted state applications in a pub ## 2. Build Guide -### 2.1 Requirements +### 2.1 Definitions -The following are **minimum** requirements to run an Aleo node: +The following snarkOS node types exist in the Aleo network: + - **Validator**: Validator nodes participate in consensus and must be started with an account that is bonded into the committee. + - **Client**: Clients do not participate in consensus but maintain a ledger. They are capable of providing information about the network as well as accepting solutions and transactions and communicating them to their peers. Two types of clients exist: + - Core client: Client node connected directly to a validator node. + - Outer Client: Client node connected only to other clients or prover nodes. + - **Prover**: Prover nodes are dedicated to solving the Aleo puzzle. They do not participate in consensus or maintain a copy of the ledger. + +### 2.2 Requirements + +The following are the requirements to run an Aleo node: - **OS**: 64-bit architectures only, latest up-to-date for security - Clients: Ubuntu 22.04 (LTS), macOS Ventura or later, Windows 11 or later - - Provers: Ubuntu 22.04 (LTS), macOS Ventura or later - Validators: Ubuntu 22.04 (LTS) - - **CPU**: 64-bit architectures only - - Clients: 16-cores - - Provers: 32-cores (64-cores preferred) - - Validators: 32-cores (64-cores preferred) + - **CPU**: 64-bit architectures only, Latest Intel Xeon or Better + - Clients, core: 32-cores (24 minimum) + - Clients, outer: 32-cores (16 minimum) + - Validators: 64-cores (or better) - **RAM**: DDR4 or better - - Clients: 16GB of memory - - Provers: 32GB of memory (64GB or larger preferred) - - Validators: 64GB of memory (128GB or larger preferred) + - Clients, core: 192GB of memory (128 minimum) + - Clients, outer: 128GB of memory (64 minimum) + - Validators: 384GB of memory (256 minimum) - **Storage**: PCIe Gen 3 x4, PCIe Gen 4 x2 NVME SSD, or better - - Clients: 64GB of disk space - - Provers: 128GB of disk space - - Validators: 2TB of disk space (4TB or larger preferred) + - Clients, core: 4TB of disk space (2 minimum) + - Clients, outer: 2TB of disk space (1 minimum) + - Validators: 6TB of disk space (4 minimum) - **Network**: Symmetric, commercial, always-on - - Clients: 100Mbps of upload **and** download bandwidth - - Provers: 250Mbps of upload **and** download bandwidth + - Clients: 250Mbps of upload **and** download bandwidth - Validators: 500Mbps of upload **and** download bandwidth -- **GPU**: - - Clients: Not required at this time - - Provers: CUDA-enabled GPU (optional) - - Validators: Not required at this time -Please note that in order to run an Aleo Prover that is **competitive**, the machine will need more than these requirements. +Please note that we do not provide official recommendations to run an Aleo Prover. In order to run an Aleo Prover that is **competitive**, the machine requirements will likely be high. -### 2.2 Installation +### 2.3 Installation Before beginning, please ensure your machine has `Rust v1.66+` installed. Instructions to [install Rust can be found here.](https://www.rust-lang.org/tools/install) @@ -93,19 +96,74 @@ cargo install --locked --path . ``` Please ensure ports `4130/tcp` and `3030/tcp` are open on your router and OS firewall. +### 2.4 Port Configuration + +#### 2.4.1 For Core Clients + +| Port | Protocol | Allow/Deny | Source | Explanation | +|----------|----------|------------|------------------------------|------------------------------------------------------------| +| 4130/tcp | TCP | Allow | All IPv4/IPv6 | TCP traffic to other clients and validators | + +#### 2.4.2 For Outer Clients + +| Port | Protocol | Allow/Deny | Source | Explanation | +|----------|----------|------------|------------------------------|------------------------------------------------------------| +| 3030/tcp | TCP | Allow | All IPv4/IPv6 | REST server | +| 4130/tcp | TCP | Allow | All IPv4/IPv6 | TCP traffic to other clients and validators | + +#### 2.4.3 For Validators + +| Port | Protocol | Allow/Deny | Source | Explanation | +|----------|----------|------------|------------------------------|------------------------------------------------------------| +| 4130/tcp | TCP | Allow | All IPv4/IPv6 | TCP traffic to other clients and validators | +| 5000/tcp | TCP | Allow | Trusted Validator IPs | TCP traffic between validators for BFT communication | +| 3000/tcp | TCP | Allow | Internal VPC or VPN | Metrics dashboard, should only be open within an internal VPC or VPN | +| 3030/tcp | TCP | Deny | All IPv4/IPv6 | REST server. This should always be disabled for validators | +| 9000/tcp | TCP | Allow | Internal VPC or VPN | Metrics export, should only be open within an internal VPC or VPN | +| 9090/tcp | TCP | Allow | Internal VPC or VPN | Prometheus metrics, should only be open within an internal VPC or VPN | ## 3. Run an Aleo Node ## 3.1 Run an Aleo Client Start by following the instructions in the [Build Guide](#2-build-guide). +The next step depends on if you want to run a `core` client or an `outer` client. Validators and related entities may want to run a `core` client. In all other cases, running an `outer` client is the default. + +### 3.1.1 Run an Aleo Core Client + +Generally, we recommend the following start command: +`snarkos start --client --nodisplay --node 0.0.0.0:4130 --peers “validator_ip:4130,core_client_ip_1:4130,core_client_ip_2:4130,core_client_ip3:4130,outer_client_ip_1:4130,...” --verbosity 1 --norest` + +To start a core client node, you can also run the following command from the `snarkOS` directory: +``` +./run-core-client.sh +``` + +### 3.1.2 Run an Aleo Outer Client + +Generally, we recommend the following start command: +`snarkos start --client --nodisplay --node 0.0.0.0:4130 --peers “validator_ip:4130,core_client_ip_1:4130,core_client_ip_2:4130,core_client_ip3:4130,outer_client_ip_1:4130,...” --verbosity 1 --rest 0.0.0.0:3030` + +To start an outer client node, you can also run the following command from the `snarkOS` directory: +``` +./run-outer-client.sh +``` + +## 3.2 Run an Aleo Validator + +Start by following the instructions in the [Build Guide](#2-build-guide). + +Generally, we recommend the following start command: +`snarkos start --validator --nodisplay --bft 0.0.0.0:5000 --node 0.0.0.0:4130 --peers “core_client_ip_1:4130,core_client_ip_2:4130,...” --validators “validator_ip_1:5000,validator_ip_2:5000,...” --verbosity 1 --norest --private-key-file ~/snarkOS/privatekey` + +Instead of specifying a private key file (`--private-key-file` flag), the private key can also be defined explicitly (`--private-key` flag). -Next, to start a client node, from the `snarkOS` directory, run: +To start a validator, you can also run the following command from the `snarkOS` directory: ``` -./run-client.sh +./run-validator.sh ``` -## 3.2 Run an Aleo Prover +## 3.3 Run an Aleo Prover Start by following the instructions in the [Build Guide](#2-build-guide). diff --git a/run-client.sh b/run-client.sh deleted file mode 100755 index ba2301f164..0000000000 --- a/run-client.sh +++ /dev/null @@ -1,40 +0,0 @@ -#!/bin/bash - -# USAGE examples: - # CLI : ./run-client.sh - -COMMAND='cargo run --release -- start --nodisplay --client' - -for word in $*; -do - COMMAND="${COMMAND} ${word}" -done - -function exit_node() -{ - echo "Exiting..." - kill $! - exit -} - -trap exit_node SIGINT - -echo "Running an Aleo client node..." -$COMMAND & - -while : -do - echo "Checking for updates..." - git stash - rm Cargo.lock - STATUS=$(git pull) - - if [ "$STATUS" != "Already up to date." ]; then - echo "Updated code found, rebuilding and relaunching client" - cargo clean - kill -INT $!; sleep 2; $COMMAND & - fi - - sleep 1800 - -done diff --git a/run-core-client.sh b/run-core-client.sh new file mode 100755 index 0000000000..bf2cf1e0e7 --- /dev/null +++ b/run-core-client.sh @@ -0,0 +1,48 @@ +#!/bin/bash + +# USAGE examples: + # CLI with env vars: PEERS=core_client_ip_1:4130,core_client_ip_2:4130,core_client_ip_3:4130,outer_client_ip_1:4130,... ./run-core-client.sh + # CLI with prompts for vars: ./run-core-client.sh + +# If the env var PEERS is not set, prompt for it +if [ -z "${PEERS}" ] +then + read -r -p "Enter the peers (comma-separated) (e.g., core_client_ip_1:4130,core_client_ip_2:4130,core_client_ip_3:4130,outer_client_ip_1:4130,...): " + PEERS=$REPLY +fi + +if [ "${PEERS}" == "" ] +then + echo "Missing peers." + exit 1 +fi + +COMMAND='cargo run --release -- start --nodisplay --client --node 0.0.0.0:4130 --peers ${PEERS} --verbosity 1 --norest' + +for word in $*; +do + COMMAND="${COMMAND} ${word}" +done + +function exit_node() +{ + echo "Exiting..." + kill $! + exit +} + +trap exit_node SIGINT + +echo "Checking for updates..." +git stash +rm Cargo.lock +STATUS=$(git pull) + +if [ "$STATUS" != "Already up to date." ]; then + echo "Updated code found, cleaning the project" + cargo clean +fi + +echo "Running an Aleo Core Client node..." +$COMMAND & +wait diff --git a/run-outer-client.sh b/run-outer-client.sh new file mode 100755 index 0000000000..5a4097d09a --- /dev/null +++ b/run-outer-client.sh @@ -0,0 +1,48 @@ +#!/bin/bash + +# USAGE examples: + # CLI with env vars: PEERS=core_client_ip_1:4130,core_client_ip_2:4130,core_client_ip_3:4130,outer_client_ip_1:4130,... ./run-outer-client.sh + # CLI with prompts for vars: ./run-outer-client.sh + +# If the env var PEERS is not set, prompt for it +if [ -z "${PEERS}" ] +then + read -r -p "Enter the peers (comma-separated) (e.g., core_client_ip_1:4130,core_client_ip_2:4130,core_client_ip_3:4130,outer_client_ip_1:4130,...): " + PEERS=$REPLY +fi + +if [ "${PEERS}" == "" ] +then + echo "Missing peers." + exit 1 +fi + +COMMAND='cargo run --release -- start --nodisplay --client --node 0.0.0.0:4130 --peers ${PEERS} --verbosity 1 --rest 0.0.0.0:3030' + +for word in $*; +do + COMMAND="${COMMAND} ${word}" +done + +function exit_node() +{ + echo "Exiting..." + kill $! + exit +} + +trap exit_node SIGINT + +echo "Checking for updates..." +git stash +rm Cargo.lock +STATUS=$(git pull) + +if [ "$STATUS" != "Already up to date." ]; then + echo "Updated code found, cleaning the project" + cargo clean +fi + +echo "Running an Aleo Outer Client node..." +$COMMAND & +wait diff --git a/run-prover.sh b/run-prover.sh index 12de37a913..22d8add045 100755 --- a/run-prover.sh +++ b/run-prover.sh @@ -32,20 +32,15 @@ function exit_node() trap exit_node SIGINT -echo "Running an Aleo Prover node..." -$COMMAND & - -while : -do - echo "Checking for updates..." - git stash - STATUS=$(git pull) +echo "Checking for updates..." +git stash +STATUS=$(git pull) - if [ "$STATUS" != "Already up to date." ]; then - echo "Updated code found, rebuilding and relaunching prover" - cargo clean - kill -INT $!; sleep 2; $COMMAND & - fi +if [ "$STATUS" != "Already up to date." ]; then + echo "Updated code found, cleaning the project" + cargo clean +fi - sleep 1800; -done +echo "Running an Aleo Prover node..." +$COMMAND & +wait diff --git a/run-validator.sh b/run-validator.sh index 44130cbee2..e46cec8014 100644 --- a/run-validator.sh +++ b/run-validator.sh @@ -1,6 +1,6 @@ #!/bin/bash # USAGE examples: - # CLI with env vars: VALIDATOR_PRIVATE_KEY=APrivateKey1... ./run-validator.sh + # CLI with env vars: VALIDATOR_PRIVATE_KEY=APrivateKey1... PEERS=core_client_ip_1:4130,core_client_ip_2:4130,... VALIDATORS=validator_ip_1:5000,validator_ip_2:5000,... ./run-validator.sh # CLI with prompts for vars: ./run-validator.sh # If the env var VALIDATOR_PRIVATE_KEY is not set, prompt for it @@ -16,7 +16,33 @@ then exit fi -COMMAND="cargo run --release -- start --nodisplay --validator --private-key ${VALIDATOR_PRIVATE_KEY}" +# If the env var PEERS is not set, prompt for it +if [ -z "${PEERS}" ] +then + read -r -p "Enter the peers (comma-separated) (e.g., core_client_ip_1:4130,core_client_ip_2:4130,...): " + PEERS=$REPLY +fi + +if [ "${PEERS}" == "" ] +then + echo "Missing peers." + exit 1 +fi + +# If the env var VALIDATORS is not set, prompt for it +if [ -z "${VALIDATORS}" ] +then + read -r -p "Enter the validators (comma-separated) (e.g., validator_ip_1:5000,validator_ip_2:5000,...): " + VALIDATORS=$REPLY +fi + +if [ "${VALIDATORS}" == "" ] +then + echo "Missing validators." + exit 1 +fi + +COMMAND="cargo run --release -- start --nodisplay --validator --bft 0.0.0.0:5000 --node 0.0.0.0:4130 --peers ${PEERS} --validators ${VALIDATORS} --norest --private-key ${VALIDATOR_PRIVATE_KEY}" for word in $*; do @@ -32,21 +58,16 @@ function exit_node() trap exit_node SIGINT +echo "Checking for updates..." +git stash +rm Cargo.lock +STATUS=$(git pull) + +if [ "$STATUS" != "Already up to date." ]; then + echo "Updated code found, cleaning the project" + cargo clean +fi + echo "Running an Aleo Validator node..." $COMMAND & - -while : -do - echo "Checking for updates..." - git stash - rm Cargo.lock - STATUS=$(git pull) - - if [ "$STATUS" != "Already up to date." ]; then - echo "Updated code found, rebuilding and relaunching validator" - cargo clean - kill -INT $!; sleep 2; $COMMAND & - fi - - sleep 1800; -done +wait From 80f722b1e77250241e72de133931af3a4ccce326 Mon Sep 17 00:00:00 2001 From: Konstantin Pandl Date: Mon, 3 Jun 2024 16:48:44 +0200 Subject: [PATCH 02/19] Update README.md Co-authored-by: Mike Turner Signed-off-by: Konstantin Pandl --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index f1fb130929..b1f2d2f7d7 100644 --- a/README.md +++ b/README.md @@ -102,7 +102,7 @@ Please ensure ports `4130/tcp` and `3030/tcp` are open on your router and OS fir | Port | Protocol | Allow/Deny | Source | Explanation | |----------|----------|------------|------------------------------|------------------------------------------------------------| -| 4130/tcp | TCP | Allow | All IPv4/IPv6 | TCP traffic to other clients and validators | +| 4130/tcp | TCP | Allow | All IPv4/IPv6 | TCP traffic to peers | #### 2.4.2 For Outer Clients From 2b0a2fb89733d1b9356c492e5b039b6fd2dfc2df Mon Sep 17 00:00:00 2001 From: Konstantin Pandl Date: Mon, 3 Jun 2024 16:48:54 +0200 Subject: [PATCH 03/19] Update README.md Co-authored-by: Mike Turner Signed-off-by: Konstantin Pandl --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index b1f2d2f7d7..f51b7cda1e 100644 --- a/README.md +++ b/README.md @@ -109,7 +109,7 @@ Please ensure ports `4130/tcp` and `3030/tcp` are open on your router and OS fir | Port | Protocol | Allow/Deny | Source | Explanation | |----------|----------|------------|------------------------------|------------------------------------------------------------| | 3030/tcp | TCP | Allow | All IPv4/IPv6 | REST server | -| 4130/tcp | TCP | Allow | All IPv4/IPv6 | TCP traffic to other clients and validators | +| 4130/tcp | TCP | Allow | All IPv4/IPv6 | TCP traffic to peers | #### 2.4.3 For Validators From a225af11fc5dc9f8dd766e46ec783069e55474fe Mon Sep 17 00:00:00 2001 From: Konstantin Pandl Date: Mon, 3 Jun 2024 16:49:02 +0200 Subject: [PATCH 04/19] Update README.md Co-authored-by: Mike Turner Signed-off-by: Konstantin Pandl --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index f51b7cda1e..acff13503d 100644 --- a/README.md +++ b/README.md @@ -115,7 +115,7 @@ Please ensure ports `4130/tcp` and `3030/tcp` are open on your router and OS fir | Port | Protocol | Allow/Deny | Source | Explanation | |----------|----------|------------|------------------------------|------------------------------------------------------------| -| 4130/tcp | TCP | Allow | All IPv4/IPv6 | TCP traffic to other clients and validators | +| 4130/tcp | TCP | Allow | All IPv4/IPv6 | TCP traffic to peers | | 5000/tcp | TCP | Allow | Trusted Validator IPs | TCP traffic between validators for BFT communication | | 3000/tcp | TCP | Allow | Internal VPC or VPN | Metrics dashboard, should only be open within an internal VPC or VPN | | 3030/tcp | TCP | Deny | All IPv4/IPv6 | REST server. This should always be disabled for validators | From e0ec38a3bc1284ee2a3df8daf013874282aca0ff Mon Sep 17 00:00:00 2001 From: Konstantin Pandl Date: Mon, 3 Jun 2024 16:49:11 +0200 Subject: [PATCH 05/19] Update README.md Co-authored-by: Mike Turner Signed-off-by: Konstantin Pandl --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index acff13503d..db2346b357 100644 --- a/README.md +++ b/README.md @@ -118,7 +118,7 @@ Please ensure ports `4130/tcp` and `3030/tcp` are open on your router and OS fir | 4130/tcp | TCP | Allow | All IPv4/IPv6 | TCP traffic to peers | | 5000/tcp | TCP | Allow | Trusted Validator IPs | TCP traffic between validators for BFT communication | | 3000/tcp | TCP | Allow | Internal VPC or VPN | Metrics dashboard, should only be open within an internal VPC or VPN | -| 3030/tcp | TCP | Deny | All IPv4/IPv6 | REST server. This should always be disabled for validators | +| 3030/tcp | TCP | Deny | All IPv4/IPv6 | REST server. This should **always** be disabled for validators | | 9000/tcp | TCP | Allow | Internal VPC or VPN | Metrics export, should only be open within an internal VPC or VPN | | 9090/tcp | TCP | Allow | Internal VPC or VPN | Prometheus metrics, should only be open within an internal VPC or VPN | From 4397f143987161e38550c24dc9ce2e0958d8b6ba Mon Sep 17 00:00:00 2001 From: Konstantin Pandl Date: Mon, 3 Jun 2024 16:49:25 +0200 Subject: [PATCH 06/19] Update README.md Co-authored-by: Mike Turner Signed-off-by: Konstantin Pandl --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index db2346b357..8d93301b3c 100644 --- a/README.md +++ b/README.md @@ -42,7 +42,7 @@ which verifies transactions and stores the encrypted state applications in a pub The following snarkOS node types exist in the Aleo network: - **Validator**: Validator nodes participate in consensus and must be started with an account that is bonded into the committee. - - **Client**: Clients do not participate in consensus but maintain a ledger. They are capable of providing information about the network as well as accepting solutions and transactions and communicating them to their peers. Two types of clients exist: + - **Client**: Clients do not participate in consensus but maintain a ledger. They are capable of providing information about the network as well as accepting solutions and transactions and communicating them to their peers. All clients run the same software, however, for the purposes of configuration management, this document defines two types of clients: - Core client: Client node connected directly to a validator node. - Outer Client: Client node connected only to other clients or prover nodes. - **Prover**: Prover nodes are dedicated to solving the Aleo puzzle. They do not participate in consensus or maintain a copy of the ledger. From a615c7c6efbf079f80f44a16701977f3627f177d Mon Sep 17 00:00:00 2001 From: Konstantin Pandl Date: Mon, 3 Jun 2024 16:49:37 +0200 Subject: [PATCH 07/19] Update README.md Co-authored-by: Mike Turner Signed-off-by: Konstantin Pandl --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 8d93301b3c..bd01a69df5 100644 --- a/README.md +++ b/README.md @@ -131,7 +131,7 @@ The next step depends on if you want to run a `core` client or an `outer` client ### 3.1.1 Run an Aleo Core Client -Generally, we recommend the following start command: +The following command is recommended when starting a client node that is connected to a validator: `snarkos start --client --nodisplay --node 0.0.0.0:4130 --peers “validator_ip:4130,core_client_ip_1:4130,core_client_ip_2:4130,core_client_ip3:4130,outer_client_ip_1:4130,...” --verbosity 1 --norest` To start a core client node, you can also run the following command from the `snarkOS` directory: From 49c4d742d60105495d87eecf414ef9768ad7d770 Mon Sep 17 00:00:00 2001 From: Konstantin Pandl Date: Mon, 3 Jun 2024 16:49:48 +0200 Subject: [PATCH 08/19] Update README.md Co-authored-by: Mike Turner Signed-off-by: Konstantin Pandl --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index bd01a69df5..88f2862be9 100644 --- a/README.md +++ b/README.md @@ -141,7 +141,7 @@ To start a core client node, you can also run the following command from the `sn ### 3.1.2 Run an Aleo Outer Client -Generally, we recommend the following start command: +The following command is recommended when starting a client node that is NOT connected to a validator: `snarkos start --client --nodisplay --node 0.0.0.0:4130 --peers “validator_ip:4130,core_client_ip_1:4130,core_client_ip_2:4130,core_client_ip3:4130,outer_client_ip_1:4130,...” --verbosity 1 --rest 0.0.0.0:3030` To start an outer client node, you can also run the following command from the `snarkOS` directory: From e22a75dba8041f06a17001c1077c4c150b89f6a4 Mon Sep 17 00:00:00 2001 From: Konstantin Pandl Date: Mon, 3 Jun 2024 16:52:03 +0200 Subject: [PATCH 09/19] Apply suggestions from code review Co-authored-by: Mike Turner Signed-off-by: Konstantin Pandl --- README.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 88f2862be9..65c3e871ac 100644 --- a/README.md +++ b/README.md @@ -69,7 +69,7 @@ The following are the requirements to run an Aleo node: - Clients: 250Mbps of upload **and** download bandwidth - Validators: 500Mbps of upload **and** download bandwidth -Please note that we do not provide official recommendations to run an Aleo Prover. In order to run an Aleo Prover that is **competitive**, the machine requirements will likely be high. +No explicit recommendations are made for proving nodes as proving hardware may be highly variable. If interested in running Aleo Provers nodes, please refer to resources published by the Aleo community. ### 2.3 Installation @@ -127,7 +127,9 @@ Please ensure ports `4130/tcp` and `3030/tcp` are open on your router and OS fir ## 3.1 Run an Aleo Client Start by following the instructions in the [Build Guide](#2-build-guide). -The next step depends on if you want to run a `core` client or an `outer` client. Validators and related entities may want to run a `core` client. In all other cases, running an `outer` client is the default. +The guide below provides information on running `core` and `outer` clients (as defined in Section 2.2.) Aleo community members running Validators are recommended to run 1-3 `core` clients as their exclusive peers. This will ensure network traffic from the public internet is verified prior to reaching the validator. + +Any client **not** connected directly to a validator can be considered an `outer` client. ### 3.1.1 Run an Aleo Core Client @@ -142,7 +144,7 @@ To start a core client node, you can also run the following command from the `sn ### 3.1.2 Run an Aleo Outer Client The following command is recommended when starting a client node that is NOT connected to a validator: -`snarkos start --client --nodisplay --node 0.0.0.0:4130 --peers “validator_ip:4130,core_client_ip_1:4130,core_client_ip_2:4130,core_client_ip3:4130,outer_client_ip_1:4130,...” --verbosity 1 --rest 0.0.0.0:3030` +`snarkos start --client --nodisplay --node 0.0.0.0:4130 --peers “core_client_ip_1:4130,core_client_ip_2:4130,core_client_ip3:4130,outer_client_ip_1:4130,...” --verbosity 1 --rest 0.0.0.0:3030` To start an outer client node, you can also run the following command from the `snarkOS` directory: ``` @@ -153,7 +155,7 @@ To start an outer client node, you can also run the following command from the ` Start by following the instructions in the [Build Guide](#2-build-guide). -Generally, we recommend the following start command: +The following command is recommended when starting a validator node: `snarkos start --validator --nodisplay --bft 0.0.0.0:5000 --node 0.0.0.0:4130 --peers “core_client_ip_1:4130,core_client_ip_2:4130,...” --validators “validator_ip_1:5000,validator_ip_2:5000,...” --verbosity 1 --norest --private-key-file ~/snarkOS/privatekey` Instead of specifying a private key file (`--private-key-file` flag), the private key can also be defined explicitly (`--private-key` flag). From 723b54184672335fb1b1e877feb7d7e498f6ae99 Mon Sep 17 00:00:00 2001 From: Konstantin Pandl Date: Mon, 3 Jun 2024 16:54:54 +0200 Subject: [PATCH 10/19] fix typo and core client script --- README.md | 2 +- run-core-client.sh | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 65c3e871ac..17741d749a 100644 --- a/README.md +++ b/README.md @@ -127,7 +127,7 @@ Please ensure ports `4130/tcp` and `3030/tcp` are open on your router and OS fir ## 3.1 Run an Aleo Client Start by following the instructions in the [Build Guide](#2-build-guide). -The guide below provides information on running `core` and `outer` clients (as defined in Section 2.2.) Aleo community members running Validators are recommended to run 1-3 `core` clients as their exclusive peers. This will ensure network traffic from the public internet is verified prior to reaching the validator. +The guide below provides information on running `core` and `outer` clients (as defined in Section 2.2.) Aleo community members running validators are recommended to run 1-3 `core` clients as their exclusive peers. This will ensure network traffic from the public internet is verified prior to reaching the validator. Any client **not** connected directly to a validator can be considered an `outer` client. diff --git a/run-core-client.sh b/run-core-client.sh index bf2cf1e0e7..e9c77cca63 100755 --- a/run-core-client.sh +++ b/run-core-client.sh @@ -1,13 +1,13 @@ #!/bin/bash # USAGE examples: - # CLI with env vars: PEERS=core_client_ip_1:4130,core_client_ip_2:4130,core_client_ip_3:4130,outer_client_ip_1:4130,... ./run-core-client.sh + # CLI with env vars: PEERS=“validator_ip:4130,core_client_ip_1:4130,core_client_ip_2:4130,core_client_ip_3:4130,outer_client_ip_1:4130,... ./run-core-client.sh # CLI with prompts for vars: ./run-core-client.sh # If the env var PEERS is not set, prompt for it if [ -z "${PEERS}" ] then - read -r -p "Enter the peers (comma-separated) (e.g., core_client_ip_1:4130,core_client_ip_2:4130,core_client_ip_3:4130,outer_client_ip_1:4130,...): " + read -r -p "Enter the peers (comma-separated) (e.g., “validator_ip:4130,core_client_ip_1:4130,core_client_ip_2:4130,core_client_ip_3:4130,outer_client_ip_1:4130,...): " PEERS=$REPLY fi From 10539241cd10103249fcf2712fc0d1a135625eec Mon Sep 17 00:00:00 2001 From: Konstantin Pandl Date: Wed, 5 Jun 2024 10:17:47 +0200 Subject: [PATCH 11/19] update validator machine checks --- cli/src/helpers/mod.rs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/cli/src/helpers/mod.rs b/cli/src/helpers/mod.rs index 7ea38c52f1..7538886e9a 100644 --- a/cli/src/helpers/mod.rs +++ b/cli/src/helpers/mod.rs @@ -91,14 +91,14 @@ pub(crate) fn check_validator_machine(node_type: NodeType) { // Retrieve the number of cores. let num_cores = num_cpus::get(); // Enforce the minimum number of cores. - let min_num_cores = 32; + let min_num_cores = 64; if num_cores < min_num_cores { let message = format!("⚠️ The number of cores ({num_cores} cores) on this machine is insufficient for a validator (minimum {min_num_cores} cores)\n"); println!("{}", message.yellow().bold()); } // Enforce the minimum amount of RAM. if let Ok(ram) = crate::helpers::detect_ram_memory() { - let min_ram = 60; + let min_ram = 238; if ram < min_ram { let message = format!("⚠️ The amount of RAM ({ram} GiB) on this machine is insufficient for a validator (minimum {min_ram} GiB)\n"); println!("{}", message.yellow().bold()); From acc940503d35fa97d0a7c9cfbcc29140fe27732d Mon Sep 17 00:00:00 2001 From: Konstantin Pandl Date: Thu, 6 Jun 2024 14:21:35 +0200 Subject: [PATCH 12/19] GiB instead of GB for RAM --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 17741d749a..2d8ad60d98 100644 --- a/README.md +++ b/README.md @@ -58,9 +58,9 @@ The following are the requirements to run an Aleo node: - Clients, outer: 32-cores (16 minimum) - Validators: 64-cores (or better) - **RAM**: DDR4 or better - - Clients, core: 192GB of memory (128 minimum) - - Clients, outer: 128GB of memory (64 minimum) - - Validators: 384GB of memory (256 minimum) + - Clients, core: 192GiB of memory (128 minimum) + - Clients, outer: 128GiB of memory (64 minimum) + - Validators: 384GiB of memory (256 minimum) - **Storage**: PCIe Gen 3 x4, PCIe Gen 4 x2 NVME SSD, or better - Clients, core: 4TB of disk space (2 minimum) - Clients, outer: 2TB of disk space (1 minimum) From a563f1e51768d15570c5e092fb4fce1cea5f61e4 Mon Sep 17 00:00:00 2001 From: Konstantin Pandl Date: Thu, 6 Jun 2024 14:30:17 +0200 Subject: [PATCH 13/19] update min RAM in GiB --- cli/src/helpers/mod.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cli/src/helpers/mod.rs b/cli/src/helpers/mod.rs index 7538886e9a..5a84cbf7d0 100644 --- a/cli/src/helpers/mod.rs +++ b/cli/src/helpers/mod.rs @@ -98,7 +98,7 @@ pub(crate) fn check_validator_machine(node_type: NodeType) { } // Enforce the minimum amount of RAM. if let Ok(ram) = crate::helpers::detect_ram_memory() { - let min_ram = 238; + let min_ram = 256; if ram < min_ram { let message = format!("⚠️ The amount of RAM ({ram} GiB) on this machine is insufficient for a validator (minimum {min_ram} GiB)\n"); println!("{}", message.yellow().bold()); From 7ea22222dcb937e79d0f5e3191cfa006305ffa3b Mon Sep 17 00:00:00 2001 From: Konstantin Pandl Date: Tue, 2 Jul 2024 00:58:18 +0200 Subject: [PATCH 14/19] update README formatting --- README.md | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 2d8ad60d98..d333d0fca8 100644 --- a/README.md +++ b/README.md @@ -54,17 +54,14 @@ The following are the requirements to run an Aleo node: - Clients: Ubuntu 22.04 (LTS), macOS Ventura or later, Windows 11 or later - Validators: Ubuntu 22.04 (LTS) - **CPU**: 64-bit architectures only, Latest Intel Xeon or Better - - Clients, core: 32-cores (24 minimum) - - Clients, outer: 32-cores (16 minimum) - - Validators: 64-cores (or better) + - Clients: 24-cores (32-cores preferred) + - Validators: 64-cores (128-cores preferred) - **RAM**: DDR4 or better - - Clients, core: 192GiB of memory (128 minimum) - - Clients, outer: 128GiB of memory (64 minimum) - - Validators: 384GiB of memory (256 minimum) + - Clients: 128GiB of memory (192GiB preferred) + - Validators: 256GiB of memory (384GiB preferred) - **Storage**: PCIe Gen 3 x4, PCIe Gen 4 x2 NVME SSD, or better - - Clients, core: 4TB of disk space (2 minimum) - - Clients, outer: 2TB of disk space (1 minimum) - - Validators: 6TB of disk space (4 minimum) + - Clients: 2TB of disk space (4TB preferred) + - Validators: 4TB of disk space (6TB preferred) - **Network**: Symmetric, commercial, always-on - Clients: 250Mbps of upload **and** download bandwidth - Validators: 500Mbps of upload **and** download bandwidth From a72232b247d0aa2b4332e7ed5d828d4ca0530696 Mon Sep 17 00:00:00 2001 From: Konstantin Pandl Date: Tue, 2 Jul 2024 00:59:44 +0200 Subject: [PATCH 15/19] fix typo --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d333d0fca8..420cdbf7ca 100644 --- a/README.md +++ b/README.md @@ -43,7 +43,7 @@ which verifies transactions and stores the encrypted state applications in a pub The following snarkOS node types exist in the Aleo network: - **Validator**: Validator nodes participate in consensus and must be started with an account that is bonded into the committee. - **Client**: Clients do not participate in consensus but maintain a ledger. They are capable of providing information about the network as well as accepting solutions and transactions and communicating them to their peers. All clients run the same software, however, for the purposes of configuration management, this document defines two types of clients: - - Core client: Client node connected directly to a validator node. + - Core Client: Client node connected directly to a validator node. - Outer Client: Client node connected only to other clients or prover nodes. - **Prover**: Prover nodes are dedicated to solving the Aleo puzzle. They do not participate in consensus or maintain a copy of the ledger. From 197cfb1984261fe4c46a0e04026b65231390d80f Mon Sep 17 00:00:00 2001 From: Konstantin Pandl Date: Tue, 2 Jul 2024 01:06:23 +0200 Subject: [PATCH 16/19] update README - add or larger preferred --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 420cdbf7ca..8faada59e4 100644 --- a/README.md +++ b/README.md @@ -54,14 +54,14 @@ The following are the requirements to run an Aleo node: - Clients: Ubuntu 22.04 (LTS), macOS Ventura or later, Windows 11 or later - Validators: Ubuntu 22.04 (LTS) - **CPU**: 64-bit architectures only, Latest Intel Xeon or Better - - Clients: 24-cores (32-cores preferred) - - Validators: 64-cores (128-cores preferred) + - Clients: 24-cores (32-cores or larger preferred) + - Validators: 64-cores (128-cores or larger preferred) - **RAM**: DDR4 or better - - Clients: 128GiB of memory (192GiB preferred) - - Validators: 256GiB of memory (384GiB preferred) + - Clients: 128GiB of memory (192GiB or larger preferred) + - Validators: 256GiB of memory (384GiB or larger preferred) - **Storage**: PCIe Gen 3 x4, PCIe Gen 4 x2 NVME SSD, or better - - Clients: 2TB of disk space (4TB preferred) - - Validators: 4TB of disk space (6TB preferred) + - Clients: 2TB of disk space (4TB or larger preferred) + - Validators: 4TB of disk space (6TB or larger preferred) - **Network**: Symmetric, commercial, always-on - Clients: 250Mbps of upload **and** download bandwidth - Validators: 500Mbps of upload **and** download bandwidth From 988f8ec2da340d684f3cc1d1066ff790de839d01 Mon Sep 17 00:00:00 2001 From: Konstantin Pandl Date: Fri, 5 Jul 2024 06:23:41 +0200 Subject: [PATCH 17/19] validators as peers --- README.md | 4 ++-- run-validator.sh | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 8faada59e4..47eb8edf5c 100644 --- a/README.md +++ b/README.md @@ -124,7 +124,7 @@ Please ensure ports `4130/tcp` and `3030/tcp` are open on your router and OS fir ## 3.1 Run an Aleo Client Start by following the instructions in the [Build Guide](#2-build-guide). -The guide below provides information on running `core` and `outer` clients (as defined in Section 2.2.) Aleo community members running validators are recommended to run 1-3 `core` clients as their exclusive peers. This will ensure network traffic from the public internet is verified prior to reaching the validator. +The guide below provides information on running `core` and `outer` clients (as defined in Section 2.2.) Aleo community members running validators are recommended to run 1-3 `core` clients as their exclusive client peers. This will ensure network traffic from the public internet is verified prior to reaching the validator. Any client **not** connected directly to a validator can be considered an `outer` client. @@ -153,7 +153,7 @@ To start an outer client node, you can also run the following command from the ` Start by following the instructions in the [Build Guide](#2-build-guide). The following command is recommended when starting a validator node: -`snarkos start --validator --nodisplay --bft 0.0.0.0:5000 --node 0.0.0.0:4130 --peers “core_client_ip_1:4130,core_client_ip_2:4130,...” --validators “validator_ip_1:5000,validator_ip_2:5000,...” --verbosity 1 --norest --private-key-file ~/snarkOS/privatekey` +`snarkos start --validator --nodisplay --bft 0.0.0.0:5000 --node 0.0.0.0:4130 --peers “validator_ip_1:4130,validator_ip_2:4130,...,core_client_ip_1:4130,core_client_ip_2:4130,...” --validators “validator_ip_1:5000,validator_ip_2:5000,...” --verbosity 1 --norest --private-key-file ~/snarkOS/privatekey` Instead of specifying a private key file (`--private-key-file` flag), the private key can also be defined explicitly (`--private-key` flag). diff --git a/run-validator.sh b/run-validator.sh index e46cec8014..742006e335 100644 --- a/run-validator.sh +++ b/run-validator.sh @@ -19,7 +19,7 @@ fi # If the env var PEERS is not set, prompt for it if [ -z "${PEERS}" ] then - read -r -p "Enter the peers (comma-separated) (e.g., core_client_ip_1:4130,core_client_ip_2:4130,...): " + read -r -p "Enter the peers (comma-separated) (e.g., validator_ip_1:4130,validator_ip_2:4130,...,core_client_ip_1:4130,core_client_ip_2:4130,...): " PEERS=$REPLY fi From 4a1b4e6bc29cb6fc0fbf8ed04a8886b085515133 Mon Sep 17 00:00:00 2001 From: Konstantin Pandl Date: Fri, 5 Jul 2024 07:18:25 +0200 Subject: [PATCH 18/19] double quotes --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 47eb8edf5c..3ddbc573ca 100644 --- a/README.md +++ b/README.md @@ -131,7 +131,7 @@ Any client **not** connected directly to a validator can be considered an `outer ### 3.1.1 Run an Aleo Core Client The following command is recommended when starting a client node that is connected to a validator: -`snarkos start --client --nodisplay --node 0.0.0.0:4130 --peers “validator_ip:4130,core_client_ip_1:4130,core_client_ip_2:4130,core_client_ip3:4130,outer_client_ip_1:4130,...” --verbosity 1 --norest` +`snarkos start --client --nodisplay --node 0.0.0.0:4130 --peers "validator_ip:4130,core_client_ip_1:4130,core_client_ip_2:4130,core_client_ip3:4130,outer_client_ip_1:4130,..." --verbosity 1 --norest` To start a core client node, you can also run the following command from the `snarkOS` directory: ``` @@ -141,7 +141,7 @@ To start a core client node, you can also run the following command from the `sn ### 3.1.2 Run an Aleo Outer Client The following command is recommended when starting a client node that is NOT connected to a validator: -`snarkos start --client --nodisplay --node 0.0.0.0:4130 --peers “core_client_ip_1:4130,core_client_ip_2:4130,core_client_ip3:4130,outer_client_ip_1:4130,...” --verbosity 1 --rest 0.0.0.0:3030` +`snarkos start --client --nodisplay --node 0.0.0.0:4130 --peers "core_client_ip_1:4130,core_client_ip_2:4130,core_client_ip3:4130,outer_client_ip_1:4130,..." --verbosity 1 --rest 0.0.0.0:3030` To start an outer client node, you can also run the following command from the `snarkOS` directory: ``` @@ -153,7 +153,7 @@ To start an outer client node, you can also run the following command from the ` Start by following the instructions in the [Build Guide](#2-build-guide). The following command is recommended when starting a validator node: -`snarkos start --validator --nodisplay --bft 0.0.0.0:5000 --node 0.0.0.0:4130 --peers “validator_ip_1:4130,validator_ip_2:4130,...,core_client_ip_1:4130,core_client_ip_2:4130,...” --validators “validator_ip_1:5000,validator_ip_2:5000,...” --verbosity 1 --norest --private-key-file ~/snarkOS/privatekey` +`snarkos start --validator --nodisplay --bft 0.0.0.0:5000 --node 0.0.0.0:4130 --peers "validator_ip_1:4130,validator_ip_2:4130,...,core_client_ip_1:4130,core_client_ip_2:4130,..." --validators "validator_ip_1:5000,validator_ip_2:5000,..." --verbosity 1 --norest --private-key-file ~/snarkOS/privatekey` Instead of specifying a private key file (`--private-key-file` flag), the private key can also be defined explicitly (`--private-key` flag). From 11b6ddbf228f97d24b17418308d48581388c7712 Mon Sep 17 00:00:00 2001 From: Konstantin Pandl Date: Mon, 22 Jul 2024 11:35:33 +0200 Subject: [PATCH 19/19] update peers handling for outer clients --- run-outer-client.sh | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/run-outer-client.sh b/run-outer-client.sh index 5a4097d09a..92f0173d98 100755 --- a/run-outer-client.sh +++ b/run-outer-client.sh @@ -11,14 +11,12 @@ then PEERS=$REPLY fi -if [ "${PEERS}" == "" ] -then - echo "Missing peers." - exit 1 +if [ -z "${PEERS}" ]; then + COMMAND="cargo run --release -- start --nodisplay --client --node 0.0.0.0:4130 --verbosity 1 --rest 0.0.0.0:3030" +else + COMMAND="cargo run --release -- start --nodisplay --client --node 0.0.0.0:4130 --peers ${PEERS} --verbosity 1 --rest 0.0.0.0:3030" fi -COMMAND='cargo run --release -- start --nodisplay --client --node 0.0.0.0:4130 --peers ${PEERS} --verbosity 1 --rest 0.0.0.0:3030' - for word in $*; do COMMAND="${COMMAND} ${word}"