Skip to content

Commit

Permalink
docs: add some info to the readme
Browse files Browse the repository at this point in the history
  • Loading branch information
wgwz committed Oct 16, 2023
1 parent ea4b12e commit aafc1a1
Showing 1 changed file with 67 additions and 2 deletions.
69 changes: 67 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,75 @@
# indexer

Blockchain indexer and database
This project indexes data from the regen ledger and stores it in a postgresql database.
It is built as single python script that spawns independent processes for indexing application specific data.
This database is currently being used for several purposes in relation to the Regen Marketplace and the Regen Groups UI.

## GraphQL API

The regen-server provides a GraphQL API for accessing the indexer database:

https://api.regen.network/indexer/v1/graphiql

Both the groups-ui and the regen-marketplace use this API to access the data.

## Dependencies

In order to run this project locally you will need the following:

1. a local testnet version of regen-ledger with REST API and RPC enabled
2. a postgresql database running
3. a working python3 installation
4. the poetry package manager for python installed

## Configuration

All configuration values are set up via environment variables, see `.env-example`.
During local development these values will be something like:

```
DATABASE_URL=postgres://postgres:postgres@localhost:5432/indexer
REGEN_RPC=http://localhost:26657
REGEN_API=http://localhost:1317
```

The `DATABASE_URL` is used both for the application to connect as well as the migration tool.

## Architecture

The main entrypoint script is `main.py`, this script spawns all other indexing processes.
The primary indexing process is contained in `index_blocks.py`, this process indexes all block-level data by looping through transactions and messages.

The scripts that index regen-specific application data are:

- `index_class_issuers.py`
- `index_proposals.py`
- `index_retires.py`
- `index_votes.py`

## Running the main script

After successfully running a local testnet version of regen-ledger and preparing a database you can run the scripts with:

```
$ poetry install
$ poetry run python main.py
```

## Managing the database with migrations

See [migrations/README.md](migrations/README.md).

## Heroku Deployment

This application is deployed in a pipeline in heroku.
There is a staging environment in addition to the production environment.

## E2E Testing

This project uses docker for e2e testing.
Currently we have a set of e2e tests written using the Regen CLI and shell scripts.
This project uses docker for e2e testing in CI.

### Instructions for running e2e tests with docker

Build docker containers (this only needs to be run once or after making changes):

Expand Down

0 comments on commit aafc1a1

Please sign in to comment.