Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added reference to helm chart usage on core,horizon install #216

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from
35 changes: 32 additions & 3 deletions docs/run-api-server/installing.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,17 @@ To install Horizon, you have a few choices. You can...
- install prebuilt binaries [from our repositories](#package-manager) via your package manager if running a Debian-based system,
- download a [prebuilt release](https://github.com/stellar/go/releases/latest) of Horizon for your target architecture and operation system, or
- [build Horizon and Stellar Core yourself](#building) from scratch.
- use [horizon helm chart](#helm-chart-installation) which internally uses same prebuilt binaries in the runtime image.

**The first method is recommended**: Not only do you ensure OS compatibility and dependency management, you'll also install some convenient wrappers that make running Horizon and Stellar Core in their respective environments much simpler.
Prebuilt binaries have several intrinsic benefits:

## Installation Methods
- Ensures OS compatibility and dependency management.
- Includes convenient wrappers that make running Horizon and Stellar Core in their respective environments much simpler.
- Cryptographic verification of the binaries.

### Package Manager

SDF publishes new releases to its custom Ubuntu repositories. Follow [this guide](https://github.com/stellar/packages/blob/master/docs/adding-the-sdf-stable-repository-to-your-system.md#adding-the-sdf-stable-repository-to-your-system) to add the stable SDF repository to your system. [This page](https://github.com/stellar/packages/blob/master/docs/installing-individual-packages.md#installing-individual-packages) outlines the various commands that these packages make available. We'll need:
SDF publishes new releases to its custom Ubuntu repositories. Follow [this guide](https://github.com/stellar/packages/blob/master/docs/adding-the-sdf-stable-repository-to-your-system.md#adding-the-sdf-stable-repository-to-your-system) to add the stable SDF repository to your system. [This page](https://github.com/stellar/packages/blob/master/docs/installing-individual-packages.md#installing-individual-packages) outlines the various commands that these packages make available. Furthermore, using apt packages has built-in checks to ensure the package contents have not been tampered with so users can guarantee that updates come from SDF:

<CodeExample>

Expand All @@ -28,6 +31,32 @@ sudo apt install stellar-horizon stellar-core

</CodeExample>

### Helm Chart Installation

If the deployment can be done on Kubernetes, there is a [horizon helm chart](https://github.com/stellar/helm-charts/blob/main/charts/horizon) available. Install the [Helm cli tool](https://helm.sh/docs/intro/install/), minimum of version 3, if you haven't already on your workstation. Next, add the Stellar repo to the helm client's list of repos and view available chart versions:

<CodeExample>

```bash
helm repo add stellar https://helm.stellar.org/charts
helm repo update stellar
helm search repo stellar/horizon --versions
```

</CodeExample>

Don't install the horizon helm chart yet, it will be done in [Running](./running.mdx), but you should review [Configuring](./configuring.mdx) first to be aware of all the potential horizon configuration parameters which the chart auto-configures by default.

If kubernetes is not an option, the helm charts may still be good reference for showing how to configure and run the horizon docker container. Just run the helm command with `template` to display the generated kubeneretes manifests which demonstrate all the container configurations needed:

<CodeExample>

```bash
helm template my-horizon stellar/horizon --values https://raw.githubusercontent.com/stellar/helm-charts/main/charts/horizon/values.yaml
```

</CodeExample>

Next, you can jump to [Testing Your Installation](#completing-and-testing-your-installation).

### Building
Expand Down
67 changes: 66 additions & 1 deletion docs/run-api-server/running.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,72 @@ sidebar_position: 40

import { CodeExample } from "@site/src/components/CodeExample";

Once your Horizon database and Captive Core configuration is set up properly, you're ready to run Horizon. Run `stellar-horizon` with the [appropriate parameters](./configuring.mdx#parameters) set (or `stellar-horizon-cmd serve` if you [installed via the package manager](./installing.mdx#package-manager), which will automatically import your configuration from `/etc/default/stellar-horizon`), which starts the HTTP server and starts logging to standard out. When run, you should see output similar to:
Once your Horizon database and Captive Core configuration is set up properly, you're ready to run Horizon.

- To run `stellar-horizon` binary direct with the [appropriate parameters](./configuring.mdx#parameters) set (or `stellar-horizon-cmd serve` if you [installed via the package manager](./installing.mdx#package-manager), which will automatically import your configuration from `/etc/default/stellar-horizon`).

- To run on Kubernetes using helm chart, ensure you have followed the [pre-requisite](./installing.mdx#helm-chart-installation) of installing the Helm cli tool and adding the Stellar chart repo to helm client:

The horizon process [requires access to a Postgres 12 database](./configuring.mdx#preparing-the-database), first use the common kubernetes cli tool `kubectl` from your workstation to create a kubernetes secret on the intended namespace of the kubernetes cluster which will hold the horizon database url.

<CodeExample>

```bash
# copy your horizon DATABASE_URL into a secure file, no line breaks.
echo -n 'database_url_here' > my_creds.txt

# now generate the kubernetes secret from the file
kubectl create secret generic \
-n my-namepsace\
my-db-secret \
--from-file=DATABASE_URL=my_creds.txt
```

</CodeExample>

Now deploy horizon onto the cluster using the helm chart:

<CodeExample>

```bash
helm install my-horizon stellar/horizon \
--namespace my-horizon-namespace-on-cluster \
--set ingest.persistence.enabled=true \
--set web.enabled=true \
--set web.existingSecret=my-db-secret \
--set global.image.horizon.tag=2.26.1 \
--set global.network=testnet \
--set ingest.existingSecret=my-db-secret \
--set ingest.horizonConfig.captiveCoreUseDb=true \
--set ingest.resources.limits.cpu=1 \
--set ingest.resources.limits.memory=6Gi
```

</CodeExample>

This example of helm chart usage, highlights some key aspects:

- uses the `global.network=[testnet|pubnet]` parameter, this automates generation of all the horizon configuration parameters specific to network such as archive urls, captive core config, etc and other parameters as mentioned on [Configuring](./configuring.mdx).
- enables all roles on the deployment instance: ingesting, api, transaction submission.
- to customize further, the best approach is to download the [horizon helm chart values.yaml](https://github.com/stellar/helm-charts/blob/main/charts/horizon/values.yaml), update the settings in your local copy of values.yaml, and pass to helm install, rather than have many individual `--set` on helm install:

<CodeExample>

```bash
helm install myhorizon stellar/horizon \
--namespace my-horizon-namespace-on-cluster \
--values values.yaml
```

</CodeExample>

- specific to customizing network configuration parameters, if you want to connect to a network other than presets of `testnet` or `pubnet`, then you won't use `global.network`, instead, use local copy of [values.yaml](https://github.com/stellar/helm-charts/blob/main/charts/horizon/values.yaml) and set `ingest.coreConfig`, and refer to [\_core-config.tpl](https://github.com/stellar/helm-charts/blob/main/charts/horizon/templates/_core-config.tpl) for example of all the key/value pairs to include.

- minimum resource limits, verify whether `LimitRange` defaults are defined on the target namespace in kubernetes for deployment, if so, ensure that the defaults provide at least minimum resource limits of `6Gi` of memory and `1` cpu. Otherwise, define the limits explicitly on the helm install via the `ingest.resources.limits.*` shown in example, to ensure the deployed pods have adequate resources.

<br />

Horizon will start HTTP server and logging to standard out. You should see output similar to:

<CodeExample>

Expand Down
52 changes: 39 additions & 13 deletions docs/run-core-node/installation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,33 @@ title: Installing
sidebar_position: 30
---

There are three ways to install Stellar Core: you can use a Docker image, use pre-built packages, or build from source. Using a Docker image is the quickest and easiest method, so it's a good choice for a lot of developers, but if you're running Stellar Core in production you may need to use packages or, if you want to sacrifice convenience for maximum control, build from source. Whichever method you use, you should make sure to install the latest [release](https://github.com/stellar/stellar-core/releases) since releases are cumulative and backwards compatible.
There are multiple ways to install Stellar Core: you can use a Docker image, pre-built packages, helm-chart, or build from source. To help choose which to use, first identify the deployment goal of [development](./installation.mdx#development-environments) or [production](./installation.mdx#production-environments) which determines best option to use.

## Docker-based installation
Regardless of installation choice, you should make sure to install the latest [release](https://github.com/stellar/stellar-core/releases) since releases are cumulative and backwards compatible.

The version number scheme that we follow is `protocol_version.release_number.patch_number` where:

`protocol_version` is the maximum protocol version supported by that release (all versions are 100% backward compatible), `release_number` is bumped when a set of new features or bug fixes not impacting the protocol are included in the release, `patch_number` is used when a critical fix has to be deployed.

### Development environments

SDF maintains a [quickstart image](https://github.com/stellar/docker-stellar-core-horizon) that bundles Stellar Core with Horizon and postgreSQL databases. It's a quick way to set up a default, non-validating, ephemeral configuration that should work for most developers.
#### Quickstart Image

In addition to SDF images, Satoshipay maintains separate Docker images for [Stellar Core](https://github.com/satoshipay/docker-stellar-core) and [Horizon](https://github.com/satoshipay/docker-stellar-horizon). The Satoshipay Stellar Core Docker image comes in a few flavors, including one with the AWS CLI installed and one with the Google Cloud SDK installed. The Horizon image supports all Horizon environment variables.
Using the Quickstart image is the quickest and easiest method, so it's a good choice for a lot of developers. SDF maintains the [quickstart image](https://github.com/stellar/docker-stellar-core-horizon) which bundles Stellar Core with Horizon and postgreSQL databases. It's a quick way to set up a default, non-validating, ephemeral configuration that should work for most developers.

In addition to SDF images, Satoshipay maintains separate Docker images for [Stellar Core](https://github.com/satoshipay/docker-stellar-core). The Satoshipay Stellar Core Docker image comes in a few flavors, including one with the AWS CLI installed and one with the Google Cloud SDK installed.

#### Installing from source

If you want to sacrifice convenience for maximum control, build from source. See the [install from source](https://github.com/stellar/stellar-core/blob/master/INSTALL.md) for build instructions.

### Production environments

The SDF also maintains a Stellar-Core-only [standalone image](https://hub.docker.com/repository/docker/stellar/stellar-core).
When running Stellar Core in production, you should consider the following deployment options: [docker image](./installation.mdx#docker-image), [helm chart](./installation.mdx#helm-chart), or [linux package](./installation.mdx#linux-package).

#### Docker Image

The SDF maintains a Stellar-Core-only [standalone image](https://hub.docker.com/repository/docker/stellar/stellar-core).

Example usage:

Expand All @@ -39,22 +53,34 @@ The image utilizes deb packages so it's possible to confirm checksum of the stel
docker run --entrypoint=/bin/sha256sum stellar/stellar-core:latest /usr/bin/stellar-core
```

## Package-based Installation
#### Linux Package

If you are using Ubuntu 18.04 LTS or later, we provide the latest stable releases of [stellar-core](https://github.com/stellar/stellar-core) and [stellar-horizon](https://github.com/stellar/go/tree/master/services/horizon) in Debian binary package format.

You may choose to install these packages individually, which offers the greatest flexibility but requires **manual** creation of the relevant configuration files and configuration of a **PostgreSQL** database.

Most people, however, choose to install the [stellar-quickstart](https://github.com/stellar/packages/blob/master/docs/quickstart.md) package which configures a **Testnet** `stellar-core` and `stellar-horizon` both backed by a local PostgreSQL database. Once installed you can easily modify either the Stellar Core configuration if you want to connect to the public network.
#### Helm Chart

## Installing from source
If the deployment can be done on Kubernetes, there is a [core helm chart](https://github.com/stellar/helm-charts/blob/main/charts/core) available. Install the [Helm cli tool](https://helm.sh/docs/intro/install/), minimum of version 3, if you haven't already on your workstation. Next, add the Stellar repo to the helm client's list of repos, and view the available chart versions:

See the [install from source](https://github.com/stellar/stellar-core/blob/master/INSTALL.md) for build instructions.
<CodeExample>

## Release version
```bash
helm repo add stellar https://helm.stellar.org/charts
helm repo update stellar
helm search repo stellar/core --versions
```

In general you should install the latest [release](https://github.com/stellar/stellar-core/releases) build. Builds are backward compatible and are cumulative.
</CodeExample>

The version number scheme that we follow is `protocol_version.release_number.patch_number` where:
Don't install the core helm chart yet, it will be done in [Running](./running-node.mdx), but you should review [Configuring](./configuring.mdx) first to be aware of all the potential configuration parameters which can be automatatically set by chart or manually overridden.

`protocol_version` is the maximum protocol version supported by that release (all versions are 100% backward compatible), `release_number` is bumped when a set of new features or bug fixes not impacting the protocol are included in the release, `patch_number` is used when a critical fix has to be deployed.
If kubernetes is not an option, the helm charts may still be good reference for showing how to configure and run the core docker container. Just run the helm command with `template` to display the generated kubeneretes manifests which demonstrate all the container configurations needed:

<CodeExample>

```bash
helm template my-core-deployment stellar/core/ --values https://raw.githubusercontent.com/stellar/helm-charts/main/charts/core/values.yaml
```

</CodeExample>
34 changes: 32 additions & 2 deletions docs/run-core-node/running-node.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,39 @@ import { CodeExample } from "@site/src/components/CodeExample";

Once you've [set up your environment](./prerequisites.mdx), [configured your node](./configuring.mdx), set up your [quorum set](./configuring.mdx#choosing-your-quorum-set), and selected archives to `get` [history from](./configuring.mdx#history), you're ready to start Stellar Core.

Use a command equivalent to:
- To run the core binary directly, use a command equivalent to:

`$ stellar-core run`
<CodeExample>

```bash
$ stellar-core run
```

</CodeExample>

- To run the core as a deployment on kubernetes, use the [Core Helm Chart](https://github.com/stellar/helm-charts/blob/main/charts/core):

Ensure you have followed the [pre-requisite](./installation.mdx#helm-chart-installation) of installing the Helm cli tool and adding the Stellar chart repo to helm client, now deploy core to the cluster:

<CodeExample>

```bash
helm install my-core-deployment stellar/core \
--namespace my-core-namespace-on-cluster \
--set core.persistence.enabled=true \
--set global.image.core.tag=19.13.1-1460.22b9bb384.focal \
--set core.resources.limits.cpu=1 \
--set core.resources.limits.memory=4Gi \
--values https://raw.githubusercontent.com/stellar/helm-charts/main/charts/core/testnet-values.yaml
```

</CodeExample>

This example of helm chart usage, shows using the [testnet-values.yaml](https://github.com/stellar/helm-charts/blob/main/charts/core/testnet-values.yaml) which defines all the configuration parameters specific to network such as archive urls, validators, etc and other parameters as mentioned on [Configuring](./configuring.mdx).

Other 'canned' network configurations are provided, `futurenet-values.yaml` and `pubnet-values.yaml`. If you want to override these network settings, or connect to a different network entirely such as a standalone, then download and edit the local copy of the values.yaml first to align settings to target network and pass that local file to helm `--values`.

Minimum resource limits, verify whether `LimitRange` defaults are defined on the target namespace in kubernetes for deployment, if so, ensure that the defaults provide at least minimum resource limits of `4Gi` of memory and `1` cpu. Otherwise, define the limits explicitly on the helm install via the `ingest.resources.limits.*` shown in example, to ensure the deployed pods have adequate resources.

At this point, you're ready to observe your node's activity as it joins the network.

Expand Down