Skip to content

Commit

Permalink
refactor docs
Browse files Browse the repository at this point in the history
  • Loading branch information
vincentsarago committed Oct 25, 2023
1 parent 9758ce4 commit 93eb17a
Show file tree
Hide file tree
Showing 9 changed files with 249 additions and 86 deletions.
18 changes: 15 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Development - Contributing

Issues and pull requests are more than welcome: https://github.com/developmentseed/eoAPI/issues

You can also start **Discussions** in https://github.com/developmentseed/eoAPI/discussions

**dev install**

```bash
Expand All @@ -19,7 +19,9 @@ python -m pip install "psycopg[binary,pool]"
python -m pip install -e runtime/eoapi/raster["test"] # or -e runtime/eoapi/stac["test"] | -e runtime/eoapi/vector["test"]
```

Note: services might have incompatible dependencies which you can resolve by using virtual environnement per service
!!! danger

Python applications might have incompatible dependencies which you can resolve by using virtual environment *per application*

**pre-commit**

Expand All @@ -28,3 +30,13 @@ This repo is set to use `pre-commit` to run *isort*, *ruff*, *pydocstring*, *bla
```bash
$ pre-commit install
```

### Open Source

You can also contribute indirectly to eoAPI by helping on the sub-modules:

- **PgSTAC** database https://github.com/stac-utils/pgstac
- **stac-fastapi**: https://github.com/stac-utils/stac-fastapi
- **titiler-pgstac**: https://github.com/stac-utils/titiler-pgstac
- **TiTiler**: https://github.com/developmentseed/titiler
- **TiPg**: https://github.com/developmentseed/tipg
25 changes: 18 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,11 +44,22 @@

## Getting started

- Clone the repository: `git clone https://github.com/developmentseed/eoAPI.git`
- Navigate to the project: `cd eoAPI`
- Run services with `docker compose up`
- Follow the [MAXAR open data demo](https://github.com/vincentsarago/MAXAR_opendata_to_pgstac) (or get inspired by the other [demos](./demo/)) to load some data into eoAPI
- Check out the [Search Viewer](http://localhost:8081/index.html), and the API documentation ([STAC Metadata](http://localhost:8081/docs), [Raster Tiles](http://localhost:8082/docs), [Vector Tiles](http://localhost:8083/api.html))
The easiest way to start exploring the different eoAPI services is with *Docker*. Clone this repository and start the multi-container *Docker* applications using `Compose`:

```
git clone https://github.com/developmentseed/eoAPI.git
cd eoAPI
docker compose up
```

Once the application are *up*, you'll need to add STAC **Collections** and **Items** to the PgSTAC database. If you don't have, you can use the follow the [MAXAR open data demo](https://github.com/vincentsarago/MAXAR_opendata_to_pgstac) (or get inspired by the other [demos](https://github.com/developmentseed/eoAPI/tree/main/demo)).

Then you can start exploring your dataset with:

- the STAC Metadata service [http://localhost:8081](http://localhost:8081)
- the Raster service [http://localhost:8082](http://localhost:8082)

If you've added vector dataset to the `public` schema in the Postgres database, they will be available through the **Vector** service at [http://localhost:8083](http://localhost:8083).

Alternatively, you may launch the application locally:
```bash
Expand All @@ -70,9 +81,9 @@ Note: services might have incompatible dependencies which you can resolve by usi

## Deployment

This repository has current runtimes that are consistently updated with new functionality.
This repository has current runtimes that are consistently updated with new functionality.

The services can be deployed locally via docker with `docker-compose up`. The official runtimes can be launched with `docker-compose -f docker-compose.yml -f docker-compose.official.yml up stac-fastapi titiler-pgstac tipg`.
The services can be deployed locally via docker with `docker-compose up`. The official runtimes can be launched with `docker compose -f docker-compose.official.yml up stac-fastapi titiler-pgstac tipg`.

Two Infrastructure as Code (IaC) repositories are available:
- [eoapi-cdk](https://github.com/developmentseed/eoapi-cdk): A set of AWS CDK constructs to deploy eoAPI services
Expand Down
10 changes: 6 additions & 4 deletions docker-compose.official.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@ version: '3'

services:
stac-fastapi:
# Note:
# the official ghcr.io/stac-utils/stac-fastapi-pgstac image uses python 3.8 and uvicorn
# which is why here we use a custom Dockerfile using python 3.11 and gunicorn
build:
context: .
dockerfile: dockerfiles/Dockerfile.stac
Expand Down Expand Up @@ -40,7 +43,7 @@ services:
# At the time of writing, rasterio and psycopg wheels are not available for arm64 arch
# so we force the image to be built with linux/amd64
platform: linux/amd64
image: ghcr.io/stac-utils/titiler-pgstac:latest
image: ghcr.io/stac-utils/titiler-pgstac:0.8.0
ports:
- "${MY_DOCKER_IP:-127.0.0.1}:8082:8082"
environment:
Expand Down Expand Up @@ -86,7 +89,7 @@ services:
- ./dockerfiles/scripts:/tmp/scripts

tipg:
image: ghcr.io/developmentseed/tipg:latest
image: ghcr.io/developmentseed/tipg:0.4.4
ports:
- "${MY_DOCKER_IP:-127.0.0.1}:8083:8083"
environment:
Expand Down Expand Up @@ -115,8 +118,7 @@ services:
- ./dockerfiles/scripts:/tmp/scripts

database:
container_name: eoapi.db
image: ghcr.io/stac-utils/pgstac:v0.7.10
image: ghcr.io/stac-utils/pgstac:v0.8.1
environment:
- POSTGRES_USER=username
- POSTGRES_PASSWORD=password
Expand Down
24 changes: 14 additions & 10 deletions docs/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,14 @@ extra:
link: "https://developmentseed.org/blog"

nav:
- Home: "index.md"
- Intro: "intro.md"
- Services: "services.md"
- Deployment: "deployment.md"
- Contributing: "contributing.md"

- Home: index.md
- Getting started:
- intro.md
- Services: services.md
- Contributing: contributing.md
- Advanced User Guide:
- Customization: customization.md
- Deployment: deployment.md

plugins:
- search
Expand All @@ -36,17 +38,17 @@ theme:
palette:
primary: custom
accent: deep orange

features:
- navigation.tabs
- navigation.tabs.sticky
- navigation.instant
- navigation.tracking
- navigation.top
- navigation.expand
- navigation.footer
- navigation.sections
favicon: img/favicon.ico
logo: img/logo.png


markdown_extensions:
- admonition
- attr_list
Expand All @@ -59,7 +61,6 @@ markdown_extensions:
- pymdownx.caret:
insert: false
- pymdownx.details
- pymdownx.emoji
- pymdownx.escapeall:
hardbreak: true
nbsp: true
Expand All @@ -73,3 +74,6 @@ markdown_extensions:
- pymdownx.tilde
- toc:
permalink: true
- pymdownx.emoji:
emoji_index: !!python/name:material.extensions.emoji.twemoji
emoji_generator: !!python/name:material.extensions.emoji.to_svg
21 changes: 15 additions & 6 deletions docs/src/contributing.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@
---
hide:
- navigation
- toc
---

Issues and pull requests are more than welcome: https://github.com/developmentseed/eoAPI/issues

You can also start **Discussions** in https://github.com/developmentseed/eoAPI/discussions

**dev install**

```bash
Expand All @@ -23,7 +20,9 @@ python -m pip install "psycopg[binary,pool]"
python -m pip install -e runtime/eoapi/raster["test"] # or -e runtime/eoapi/stac["test"] | -e runtime/eoapi/vector["test"]
```

Note: services might have incompatible dependencies which you can resolve by using virtual environnement per service
!!! danger

Python applications might have incompatible dependencies which you can resolve by using virtual environment *per application*

**pre-commit**

Expand All @@ -32,3 +31,13 @@ This repo is set to use `pre-commit` to run *isort*, *ruff*, *pydocstring*, *bla
```bash
$ pre-commit install
```

### Open Source

You can also contribute indirectly to eoAPI by helping on the sub-modules:

- **PgSTAC** database https://github.com/stac-utils/pgstac
- **stac-fastapi**: https://github.com/stac-utils/stac-fastapi
- **titiler-pgstac**: https://github.com/stac-utils/titiler-pgstac
- **TiTiler**: https://github.com/developmentseed/titiler
- **TiPg**: https://github.com/developmentseed/tipg
67 changes: 67 additions & 0 deletions docs/src/customization.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@

The eoAPI repository (https://github.com/developmentseed/eoAPI) hosts customized version of each base services. This is to show how each services can be customized and how they can work in parallel or with each other.

---
## eoapi.stac

Built on [stac-fastapi.pgstac](https://github.com/stac-utils/stac-fastapi) application, adding a **`TiTilerExtension`** and a simple **`Search Viewer`**.

The service includes:

- Full **stac-fastapi** implementation - see [docs](http://localhost:8081/docs) if using the `docker-compose` configuration.
- Simple STAC Search **viewer** - see [viewer](http://localhost:8081/index.html) if using the `docker-compose` configuration.
- **Proxy** to the tiler endpoint for STAC Items.

When the `TITILER_ENDPOINT` environment variable is set (pointing to the `raster` application), additional endpoints will be added to the stac-fastapi application (see: [stac/extension.py](https://github.com/developmentseed/eoAPI/blob/main/src/eoapi/stac/eoapi/stac/extension.py)):

- `/collections/{collectionId}/items/{itemId}/tilejson.json`: Return the `raster` tilejson for an item
- `/collections/{collectionId}/items/{itemId}/viewer`: Redirect to the `raster` viewer

<p align="center">
<img alt="eoapi.stac OpenAPI documentation"src="https://github.com/developmentseed/eoAPI/assets/10407788/f7fc4e7f-079a-4007-8c4c-74ff9ca7b012">
<img alt="Metadata STAC search viewer" src="https://github.com/developmentseed/eoAPI/assets/10407788/b1fd6aa8-aab9-4d58-9ad4-6e1069ed9473"/>
</p>

Code: [/runtime/eoapi/stac](https://github.com/developmentseed/eoAPI/tree/main/runtime/eoapi/stac)

---
## eoapi.raster

The dynamic tiler deployed within `eoAPI` is built on top of [titiler-pgstac](https://github.com/stac-utils/titiler-pgstac) and [pgstac](https://github.com/stac-utils/pgstac). It enables large-scale mosaic based on the results of STAC search queries.

The service includes all the default endpoints from **titiler-pgstac** application and:

- `/`: a custom landing page with links to the different endpoints

- `/mosaic/builder`: a virtual mosaic builder UI, which help creating and registering STAC Search queries

- `/collections`: a *secret* (not in OpenAPI documentation) endpoint used in the mosaic-builder page

- `/collections/{collection_id}/items/{item_id}/viewer`: a simple STAC Item viewer

<p align="center">
<img alt="eoapi.stac OpenAPI documentation" src="https://github.com/developmentseed/eoAPI/assets/10407788/aa08d796-f3b1-4f43-8021-cbddfff95d26">
<img alt="Raster mosaic builder" src="https://github.com/developmentseed/eoAPI/assets/10407788/8f0c96e6-d80a-46ef-a34f-66e59c64a027"/>
<img alt="STAC Item viewer" src="https://github.com/developmentseed/eoAPI/assets/10407788/be86f07e-4b05-4c78-87bf-8738daf1ee20">
</p>


Code: [/runtime/eoapi/raster](https://github.com/developmentseed/eoAPI/tree/main/runtime/eoapi/raster)

---
## eoapi.vector

OGC Features and Tiles API built on top of [tipg](https://github.com/developmentseed/tipg).

By default, the API will look for tables in the `public` schema of the database. We've also added three functions that connect to the pgSTAC schema:

- **pg_temp.pgstac_collections_view**: Simple function which returns PgSTAC Collections
- **pg_temp.pgstac_hash**: Return features for a specific `searchId` (hash)
- **pg_temp.pgstac_hash_count**: Return the number of items per geometry for a specific `searchId` (hash)

<p align="center">
<img alt="eoapi.vector OpenAPI documentation" src="https://github.com/developmentseed/eoAPI/assets/10407788/80ff9659-bc07-4238-a94a-7cab0a2af380">
<img alt="eoapi.vector landing page" src="https://github.com/developmentseed/eoAPI/assets/10407788/b2a8a8d4-d3a1-464a-8b1a-166499ee4abd">
</p>

Code: [/runtime/eoapi/vector](https://github.com/developmentseed/eoAPI/tree/main/runtime/eoapi/vector)
24 changes: 12 additions & 12 deletions docs/src/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,11 +102,11 @@ If you don't have a k8s cluster set up on AWS or GCP then follow an IaC guide be
> &#9432; The helm chart in this repo assumes your cluster has a few third-party add-ons and controllers installed. So
> it's in your best interest to read through the IaC guides to understand what those defaults are
* [AWS EKS Cluster Setup](./docs/aws-eks.md)
* :octicons-link-external-16: [AWS EKS Cluster Setup](https://github.com/developmentseed/eoapi-k8s/blob/main/docs/aws-eks.md)
* [TBD: GCP GKE Cluster Setup](./docs/gcp-gke.md)
**Helm Installation**
* :octicons-link-external-16: [TBD: GCP GKE Cluster Setup](https://github.com/developmentseed/eoapi-k8s/blob/main/docs/gcp-gke.md)
**Helm Installation**
Once you have a k8s cluster set up you can `helm install` eoAPI as follows
Expand All @@ -121,7 +121,7 @@ Once you have a k8s cluster set up you can `helm install` eoAPI as follows
$ export POSTGRES_USER=s00pers3cr3t
$ export POSTGRES_PASSWORD=superuserfoobar
$ export PGPASSWORD=foobar
$ cd ./helm-chart
$ helm install \
Expand All @@ -141,23 +141,23 @@ Once you have a k8s cluster set up you can `helm install` eoAPI as follows
```python
# add the eoapi helm repo locally
$ helm repo add eoapi https://devseed.com/eoapi-k8s/
# list out the eoapi chart versions
$ helm search repo eoapi
NAME CHART VERSION APP VERSION DESCRIPTION
$ helm search repo eoapi
NAME CHART VERSION APP VERSION DESCRIPTION
eoapi/eoapi 0.1.1 0.1.0 Create a full Earth Observation API with Metada...
eoapi/eoapi 0.1.2 0.1.0 Create a full Earth Observation API with Metada...
# add the required secret overrides to an arbitrarily named `.yaml` file (`config.yaml` below)
$ cat config.yaml
$ cat config.yaml
db:
settings:
secrets:
PGUSER: "username"
POSTGRES_USER: "username"
PGPASSWORD: "password"
POSTGRES_PASSWORD: "password"
# then run `helm install` with those overrides
# then run `helm install` with those overrides
helm install eoapi eoapi/eoapi --version 0.1.1 -f config.yaml
```
33 changes: 22 additions & 11 deletions docs/src/intro.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,3 @@
---
hide:
- navigation
---


<p align="center">
<img width="700" src="../img/eoAPI.png"/>
Expand Down Expand Up @@ -67,11 +62,25 @@ See [service details](./services.md) for more information.

## Getting started

- Clone the repository: `git clone https://github.com/developmentseed/eoAPI.git`
- Navigate to the project: `cd eoAPI`
- Run services with `docker compose up`
- Follow the [MAXAR open data demo](https://github.com/vincentsarago/MAXAR_opendata_to_pgstac) (or get inspired by the other [demos](https://github.com/developmentseed/eoAPI/tree/main/demo)) to load some data into eoAPI
- Checkout the [Search Viewer](http://localhost:8081/index.html), and the API documentation ([STAC Metadata](http://localhost:8081/docs), [Raster Tiles](http://localhost:8082/docs), [Vector Tiles](http://localhost:8083/api.html))
The easiest way to start exploring the different eoAPI services is with *Docker*. Clone this repository and start the multi-container *Docker* applications using `Compose`:

```
git clone https://github.com/developmentseed/eoAPI.git
cd eoAPI
docker compose up
```

Once the application are *up*, you'll need to add STAC **Collections** and **Items** to the PgSTAC database. If you don't have, you can use the follow the [MAXAR open data demo](https://github.com/vincentsarago/MAXAR_opendata_to_pgstac) (or get inspired by the other [demos](https://github.com/developmentseed/eoAPI/tree/main/demo)).


Then you can start exploring your dataset with:

- the STAC Metadata service [http://localhost:8081](http://localhost:8081)
- the Raster service [http://localhost:8082](http://localhost:8082)

!!! info

If you've added vector dataset to the `public` schema in the Postgres database, they will be available through the **Vector** service at [http://localhost:8083](http://localhost:8083).

Alternatively, you may launch the application locally:
```bash
Expand All @@ -87,5 +96,7 @@ export DATABASE_URL=postgresql://username:[email protected]:5439/postgis # Conne
.venv/bin/uvicorn eoapi.{SERVICE}.app:app --port 8000 --reload
```

Note: services might have incompatible dependencies which you can resolve by using virtual environement per service
!!! danger

Python applications might have incompatible dependencies which you can resolve by using virtual environment *per application*

Loading

0 comments on commit 93eb17a

Please sign in to comment.