---
-**Documentation**: https://eoapi.dev
-
-**Source Code**: https://github.com/developmentseed/eoAPI
+- **Documentation**: https://eoapi.dev
+- **Source Code**: https://github.com/developmentseed/eoAPI
---
-## **E**arth **O**bservation **API**
-
-`eoAPI` combines several *state-of-the-art* projects to create a full Earth Observation API. Each service can be used and deployed independently, but `eoAPI` creates the interconnections between each service:
-
-- **pgSTAC** database [https://github.com/stac-utils/pgstac](https://github.com/stac-utils/pgstac)
+## Earth Observation API
-- **STAC API** built on top of [https://github.com/stac-utils/stac-fastapi](https://github.com/stac-utils/stac-fastapi)
+`eoAPI` combines several state-of-the-art projects to create a full Earth Observation API. Each service can be used and deployed independently, but `eoAPI` creates the interconnections between each service:
-- **STAC Items And Mosaic Raster Tiles** API built on top of [https://github.com/stac-utils/titiler-pgstac](https://github.com/stac-utils/titiler-pgstac)
-
-- **OGC Features and Vector Tiles** API built on top of [https://github.com/developmentseed/tipg](https://github.com/developmentseed/tipg)
-
-- **A STAC Catalog browsing UI** based on the radiant earth browser : [https://github.com/radiantearth/stac-browser](https://github.com/radiantearth/stac-browser)
+- **pgSTAC** database: https://github.com/stac-utils/pgstac
+- **STAC API** built on top of: https://github.com/stac-utils/stac-fastapi
+- **STAC Items And Mosaic Raster Tiles** API built on top of: https://github.com/stac-utils/titiler-pgstac
+- **OGC Features and Vector Tiles** API built on top of: https://github.com/developmentseed/tipg
+- **A STAC Catalog browsing UI** based on the radiant earth browser: https://github.com/radiantearth/stac-browser
---
@@ -40,38 +33,33 @@
`eoAPI` is proudly open-source and driven by a dedicated community of contributors. We believe in the power of open collaboration and welcome anyone to contribute, discuss, and grow this tool. Join the conversations on [GitHub Discussions](https://github.com/developmentseed/eoAPI/discussions) and make a difference in the Earth Observation realm.
-
---
## Getting started
-The easiest way to start exploring the different eoAPI services is with *Docker*. Clone this repository and start the multi-container *Docker* applications using `Compose`:
+The easiest way to explore the different eoAPI services is with **Docker**. Clone this repository and start the multi-container **Docker** applications using `Compose`:
-```
+```bash
git clone https://github.com/developmentseed/eoAPI.git
cd eoAPI
docker compose up
```
-Once the applications are *up*, you'll need to add STAC **Collections** and **Items** to the PgSTAC database. If you don't have these available, you can follow the [MAXAR open data demo](https://github.com/vincentsarago/MAXAR_opendata_to_pgstac) (or get inspired by the other [demos](https://github.com/developmentseed/eoAPI/tree/main/demo)).
+Once the applications are **up**, you'll need to add STAC **Collections** and **Items** to the PgSTAC database. If you don't have these available, you can follow the [MAXAR open data demo](https://github.com/vincentsarago/MAXAR_opendata_to_pgstac) (or get inspired by the other [demos](https://github.com/developmentseed/eoAPI/tree/main/demo)).
Then you can start exploring your dataset with:
- - the STAC Metadata service [http://localhost:8081](http://localhost:8081)
- - the Raster service [http://localhost:8082](http://localhost:8082)
- - the browser UI [http://localhost:8085](http://localhost:8085)
-
-If you've added a vector dataset to the `public` schema in the Postgres database, they will be available through the **Vector** service at [http://localhost:8083](http://localhost:8083).
-
-## Deployment with standard runtimes
+- the STAC Metadata service: http://localhost:8081
+- the Raster service: http://localhost:8082
+- the browser UI: http://localhost:8085
-This repository has current runtimes that are consistently updated with new functionality.
+If you've added a vector dataset to the `public` schema in the Postgres database, they will be available through the **Vector** service at http://localhost:8083.
### Local deployment
The services can be deployed altogether locally with `docker compose up`.
-Alternatively, you may install the libraries and launch the applications manually:
+Alternatively, you may install the libraries and launch the applications manually:
@@ -79,29 +67,27 @@ Alternatively, you may install the libraries and launch the applications manuall
python -m pip install --upgrade virtualenv
virtualenv .venv
source .venv/bin/activate
-
export DATABASE_URL=postgresql://username:password@0.0.0.0:5439/postgis # Connect to the database of your choice
-
python -m pip install uvicorn
###############################################################################
# Install and launch the application
# Select one of the following
-
###############################################################################
+
# STAC
python -m pip install "psycopg[binary,pool]" stac-fastapi-pgstac
-.venv/bin/uvicorn stac_fastapi.pgstac.app:app --port 8081 --reload
+python -m uvicorn stac_fastapi.pgstac.app:app --port 8081 --reload
###############################################################################
# RASTER
python -m pip install "psycopg[binary,pool]" titiler-pgstac
-.venv/bin/uvicorn titiler.pgstac.main:app --port 8082 --reload
+python -m uvicorn titiler.pgstac.main:app --port 8082 --reload
###############################################################################
# VECTOR
python -m pip install tipg
-.venv/bin/uvicorn tipg.main:app --port 8083 --reload
+python -m uvicorn tipg.main:app --port 8083 --reload
```
Note: Python libraries might have incompatible dependencies, which you can resolve by using a virtual environment for each one.
@@ -110,51 +96,27 @@ Note: Python libraries might have incompatible dependencies, which you can resol
### Deployment on the cloud
-#### Kubernetes
+#### Kubernetes
[eoapi-k8s](https://github.com/developmentseed/eoapi-k8s) contains IaC and Helm charts for deploying eoAPI services on AWS and GCP.
#### AWS CDK
-[eoapi-cdk](https://github.com/developmentseed/eoapi-cdk) defines a set of AWS CDK constructs that can be used to deploy eoAPI services on AWS. This repository itself makes use of these in `infrastructure/aws`. An official example usage of these constructs can be found at [eoapi-template](https://github.com/developmentseed/eoapi-template).
-
-
+[eoapi-cdk](https://github.com/developmentseed/eoapi-cdk) defines a set of AWS CDK constructs that can be used to deploy eoAPI services on AWS. An official example of these constructs is at [eoapi-template](https://github.com/developmentseed/eoapi-template).
## Deployment with custom runtimes
-The eoAPI repository hosts customized versions of each base service which can work in parallel or in combination with each other.
-
-eoAPI custom runtimes can be launched with docker:
-
-```
-docker compose -f docker-compose.custom.yml --profile gunicorn up
-```
-
-Alternatively, you may launch the application locally:
-```bash
-python -m pip install --upgrade virtualenv
-virtualenv .venv
-source .venv/bin/activate
-
-python -m pip install "psycopg[binary,pool]" uvicorn
-python -m pip install runtime/eoapi/{SERVICE} # SERVICE should be one of `raster, vector, stac.`
-
-export DATABASE_URL=postgresql://username:password@0.0.0.0:5439/postgis # Connect to the database of your choice
-
-.venv/bin/uvicorn eoapi.{SERVICE}.app:app --port 8000 --reload
-```
-
-Note: services might have incompatible dependencies, which you can resolve by using a virtual environment for each service.
+An example of custom eoAPI runtimes and deployment can be found at [eoapi-devseed](https://github.com/developmentseed/eoapi-devseed).
## Contribution & Development
-We highly value and rely on our community! You can make a difference whether you're an expert or just getting started. Here's how:
+We highly value and rely on our community! Whether you're an expert or just getting started, you can make a difference. Here's how:
-- **Contribute**: Check out our [CONTRIBUTING.md](https://github.com/developmentseed/eoAPI/blob/main/CONTRIBUTING.md) guide to understand how you can contribute.
- **Engage in Discussions**: Share your ideas, ask questions, or provide feedback through [GitHub Discussions](https://github.com/developmentseed/eoAPI/discussions). This is where most of our project conversations take place.
- **Report Issues**: Found a bug or have a feature request? Raise it on our [issues page](https://github.com/developmentseed/eoAPI/issues).
---
+
## License
At Development Seed, we believe in open collaboration and making tools and data more accessible. In line with this ethos, we've explicitly chosen a MIT license for `eoAPI`.
@@ -163,10 +125,6 @@ For full license details, see [LICENSE](https://github.com/developmentseed/eoAPI
## Authors
-Nurtured by [Development Seed]()
-
-See [contributors](https://github.com/developmentseed/eoAPI/graphs/contributors) for a listing of individual contributors.
-
-## Changes
+Nurtured by [Development Seed](http://developmentseed.org)
-See [CHANGES.md](https://github.com/developmentseed/eoAPI/blob/main/CHANGES.md).
+See [contributors](https://github.com/developmentseed/eoAPI/graphs/contributors) for a listing of individual contributors.
\ No newline at end of file
diff --git a/docker-compose.custom.yml b/docker-compose.custom.yml
deleted file mode 100644
index 2da28e5..0000000
--- a/docker-compose.custom.yml
+++ /dev/null
@@ -1,270 +0,0 @@
-version: '3'
-
-services:
- stac-browser:
- profiles:
- - gunicorn
- build:
- context: dockerfiles
- dockerfile: Dockerfile.browser
- ports:
- - "${MY_DOCKER_IP:-127.0.0.1}:8085:8085"
- depends_on:
- - stac
- - database
- - raster
- stac:
- container_name: eoapi.stac
- profiles:
- - gunicorn
- build:
- context: .
- dockerfile: dockerfiles/Dockerfile.stac
- ports:
- - "${MY_DOCKER_IP:-127.0.0.1}:8081:8081"
- environment:
- - APP_HOST=0.0.0.0
- - APP_PORT=8081
- - HOST=0.0.0.0
- - PORT=8081
- - ENVIRONMENT=local
- # https://github.com/tiangolo/uvicorn-gunicorn-docker#web_concurrency
- - WEB_CONCURRENCY=${WEB_CONCURRENCY:-10}
- # Postgres connection
- - POSTGRES_USER=username
- - POSTGRES_PASS=password
- - POSTGRES_DBNAME=postgis
- - POSTGRES_HOST_READER=database
- - POSTGRES_HOST_WRITER=database
- - POSTGRES_PORT=5432
- - DB_MIN_CONN_SIZE=1
- - DB_MAX_CONN_SIZE=10
- # https://github.com/developmentseed/eoAPI/issues/16
- # - TITILER_ENDPOINT=raster
- - TITILER_ENDPOINT=http://127.0.0.1:8082
- # PgSTAC extensions
- # - EOAPI_STAC_EXTENSIONS=["filter", "query", "sort", "fields", "pagination", "context", "transaction"]
- # - EOAPI_STAC_CORS_METHODS='GET,POST,PUT,OPTIONS'
- depends_on:
- - database
- - raster
- command:
- bash -c "bash /tmp/scripts/wait-for-it.sh -t 120 -h database -p 5432 && /start.sh"
- volumes:
- - ./dockerfiles/scripts:/tmp/scripts
-
- raster:
- container_name: eoapi.raster
- profiles:
- - gunicorn
- build:
- context: .
- dockerfile: dockerfiles/Dockerfile.raster
- # At the time of writing, rasterio and psycopg wheels are not available for arm64 arch
- # so we force the image to be built with linux/amd64
- platform: linux/amd64
- ports:
- - "${MY_DOCKER_IP:-127.0.0.1}:8082:8082"
- environment:
- # Application
- - HOST=0.0.0.0
- - PORT=8082
- # https://github.com/tiangolo/uvicorn-gunicorn-docker#web_concurrency
- - WEB_CONCURRENCY=${WEB_CONCURRENCY:-10}
- # Postgres connection
- - POSTGRES_USER=username
- - POSTGRES_PASS=password
- - POSTGRES_DBNAME=postgis
- - POSTGRES_HOST=database
- - POSTGRES_PORT=5432
- - DB_MIN_CONN_SIZE=1
- - DB_MAX_CONN_SIZE=10
- # - DB_MAX_QUERIES=10
- # - DB_MAX_IDLE=10
- # GDAL Config
- - CPL_TMPDIR=/tmp
- - GDAL_CACHEMAX=75%
- - GDAL_INGESTED_BYTES_AT_OPEN=32768
- - GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR
- - GDAL_HTTP_MERGE_CONSECUTIVE_RANGES=YES
- - GDAL_HTTP_MULTIPLEX=YES
- - GDAL_HTTP_VERSION=2
- - VSI_CACHE=TRUE
- - VSI_CACHE_SIZE=536870912
- # TiTiler Config
- - MOSAIC_CONCURRENCY=1
- # AWS S3 endpoint config
- - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
- - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
- # API Config
- - EOAPI_RASTER_ENABLE_MOSAIC_SEARCH=TRUE
- depends_on:
- - database
- command:
- bash -c "bash /tmp/scripts/wait-for-it.sh -t 120 -h database -p 5432 && /start.sh"
- volumes:
- - ./dockerfiles/scripts:/tmp/scripts
-
- vector:
- container_name: eoapi.vector
- profiles:
- - gunicorn
- build:
- context: .
- dockerfile: dockerfiles/Dockerfile.vector
- ports:
- - "${MY_DOCKER_IP:-127.0.0.1}:8083:8083"
- environment:
- # Application
- - HOST=0.0.0.0
- - PORT=8083
- # https://github.com/tiangolo/uvicorn-gunicorn-docker#web_concurrency
- - WEB_CONCURRENCY=${WEB_CONCURRENCY:-10}
- # Postgres connection
- - POSTGRES_USER=username
- - POSTGRES_PASS=password
- - POSTGRES_DBNAME=postgis
- - POSTGRES_HOST=database
- - POSTGRES_PORT=5432
- - DB_MIN_CONN_SIZE=1
- - DB_MAX_CONN_SIZE=10
- command:
- bash -c "bash /tmp/scripts/wait-for-it.sh -t 120 -h database -p 5432 && /start.sh"
- depends_on:
- - database
- volumes:
- - ./dockerfiles/scripts:/tmp/scripts
-
- stac-uvicorn:
- container_name: eoapi.stac-uvicorn
- profiles:
- - uvicorn
- build:
- context: .
- dockerfile: dockerfiles/Dockerfile.stac-uvicorn
- ports:
- - "${MY_DOCKER_IP:-127.0.0.1}:8081:8081"
- environment:
- - APP_HOST=0.0.0.0
- - APP_PORT=8081
- - HOST=0.0.0.0
- - PORT=8081
- - ENVIRONMENT=local
- # Postgres connection
- - POSTGRES_USER=username
- - POSTGRES_PASS=password
- - POSTGRES_DBNAME=postgis
- - POSTGRES_HOST_READER=database
- - POSTGRES_HOST_WRITER=database
- - POSTGRES_PORT=5432
- - DB_MIN_CONN_SIZE=1
- - DB_MAX_CONN_SIZE=10
- # https://github.com/developmentseed/eoAPI/issues/16
- # - TITILER_ENDPOINT=raster
- - TITILER_ENDPOINT=http://127.0.0.1:8082
- # PgSTAC extensions
- # - EOAPI_STAC_EXTENSIONS=["filter", "query", "sort", "fields", "pagination", "context", "transaction"]
- # - EOAPI_STAC_CORS_METHODS='GET,POST,PUT,OPTIONS'
- depends_on:
- - database
- - raster-uvicorn
- command:
- bash -c "bash /tmp/scripts/wait-for-it.sh -t 120 -h database -p 5432 && uvicorn eoapi.stac.app:app --host 0.0.0.0 --port 8081"
- volumes:
- - ./dockerfiles/scripts:/tmp/scripts
-
- raster-uvicorn:
- container_name: eoapi.raster-uvicorn
- profiles:
- - uvicorn
- build:
- context: .
- dockerfile: dockerfiles/Dockerfile.raster-uvicorn
- # At the time of writing, rasterio and psycopg wheels are not available for arm64 arch
- # so we force the image to be built with linux/amd64
- platform: linux/amd64
- ports:
- - "${MY_DOCKER_IP:-127.0.0.1}:8082:8082"
- environment:
- # Application
- - HOST=0.0.0.0
- - PORT=8082
- # Postgres connection
- - POSTGRES_USER=username
- - POSTGRES_PASS=password
- - POSTGRES_DBNAME=postgis
- - POSTGRES_HOST=database
- - POSTGRES_PORT=5432
- - DB_MIN_CONN_SIZE=1
- - DB_MAX_CONN_SIZE=10
- # GDAL Config
- - CPL_TMPDIR=/tmp
- - GDAL_CACHEMAX=75%
- - GDAL_INGESTED_BYTES_AT_OPEN=32768
- - GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR
- - GDAL_HTTP_MERGE_CONSECUTIVE_RANGES=YES
- - GDAL_HTTP_MULTIPLEX=YES
- - GDAL_HTTP_VERSION=2
- - VSI_CACHE=TRUE
- - VSI_CACHE_SIZE=536870912
- # TiTiler Config
- - MOSAIC_CONCURRENCY=1
- # AWS S3 endpoint config
- - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
- - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
- # API Config
- - EOAPI_RASTER_ENABLE_MOSAIC_SEARCH=TRUE
- depends_on:
- - database
- command:
- bash -c "bash /tmp/scripts/wait-for-it.sh -t 120 -h database -p 5432 && uvicorn eoapi.raster.app:app --host 0.0.0.0 --port 8082"
- volumes:
- - ./dockerfiles/scripts:/tmp/scripts
-
- vector-uvicorn:
- container_name: eoapi.vector-uvicorn
- profiles:
- - uvicorn
- build:
- context: .
- dockerfile: dockerfiles/Dockerfile.vector-uvicorn
- ports:
- - "${MY_DOCKER_IP:-127.0.0.1}:8083:8083"
- environment:
- # Application
- - HOST=0.0.0.0
- - PORT=8083
- # Postgres connection
- - POSTGRES_USER=username
- - POSTGRES_PASS=password
- - POSTGRES_DBNAME=postgis
- - POSTGRES_HOST=database
- - POSTGRES_PORT=5432
- - DB_MIN_CONN_SIZE=1
- - DB_MAX_CONN_SIZE=10
- command:
- bash -c "bash /tmp/scripts/wait-for-it.sh -t 120 -h database -p 5432 && uvicorn eoapi.vector.app:app --host 0.0.0.0 --port 8083"
- depends_on:
- - database
- volumes:
- - ./dockerfiles/scripts:/tmp/scripts
-
- database:
- container_name: eoapi.db
- image: ghcr.io/stac-utils/pgstac:v0.8.1
- environment:
- - POSTGRES_USER=username
- - POSTGRES_PASSWORD=password
- - POSTGRES_DB=postgis
- - PGUSER=username
- - PGPASSWORD=password
- - PGDATABASE=postgis
- ports:
- - "${MY_DOCKER_IP:-127.0.0.1}:5439:5432"
- command: postgres -N 500
- volumes:
- - ./.pgdata:/var/lib/postgresql/data
-
-networks:
- default:
- name: eoapi-network
diff --git a/docker-compose.yml b/docker-compose.yml
index f43fec0..4b944a5 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -13,29 +13,12 @@ services:
- stac-fastapi
- titiler-pgstac
- database
-
+
stac-fastapi:
- # Note:
- # the official ghcr.io/stac-utils/stac-fastapi-pgstac image uses python 3.8 and uvicorn
- # which is why here we use a custom Dockerfile using python 3.11 and gunicorn
- build:
- context: .
- dockerfile: dockerfiles/Dockerfile.stac
+ image: ghcr.io/stac-utils/stac-fastapi-pgstac:latest
ports:
- "${MY_DOCKER_IP:-127.0.0.1}:8081:8081"
environment:
- # Application
- - HOST=0.0.0.0
- - PORT=8081
- - MODULE_NAME=stac_fastapi.pgstac.app
- - VARIABLE_NAME=app
- # gunicorn
- # https://github.com/tiangolo/uvicorn-gunicorn-docker#web_concurrency
- - WEB_CONCURRENCY=10
- # https://github.com/tiangolo/uvicorn-gunicorn-docker#workers_per_core
- # - WORKERS_PER_CORE=1
- # https://github.com/tiangolo/uvicorn-gunicorn-docker#max_workers
- # - MAX_WORKERS=10
# Postgres connection
- POSTGRES_USER=username
- POSTGRES_PASS=password
@@ -44,11 +27,11 @@ services:
- POSTGRES_HOST_WRITER=database
- POSTGRES_PORT=5432
- DB_MIN_CONN_SIZE=1
- - DB_MAX_CONN_SIZE=10
+ - DB_MAX_CONN_SIZE=1
depends_on:
- database
command:
- bash -c "bash /tmp/scripts/wait-for-it.sh -t 120 -h database -p 5432 && /start.sh"
+ bash -c "bash /tmp/scripts/wait-for-it.sh -t 120 -h database -p 5432 && uvicorn stac_fastapi.pgstac.app:app --host 0.0.0.0 --port 8081"
volumes:
- ./dockerfiles/scripts:/tmp/scripts
@@ -56,7 +39,7 @@ services:
# At the time of writing, rasterio and psycopg wheels are not available for arm64 arch
# so we force the image to be built with linux/amd64
platform: linux/amd64
- image: ghcr.io/stac-utils/titiler-pgstac:1.0.0a3
+ image: ghcr.io/stac-utils/titiler-pgstac:1.3.0
ports:
- "${MY_DOCKER_IP:-127.0.0.1}:8082:8082"
environment:
@@ -102,7 +85,7 @@ services:
- ./dockerfiles/scripts:/tmp/scripts
tipg:
- image: ghcr.io/developmentseed/tipg:0.5.0
+ image: ghcr.io/developmentseed/tipg:0.7.1
ports:
- "${MY_DOCKER_IP:-127.0.0.1}:8083:8083"
environment:
@@ -131,7 +114,7 @@ services:
- ./dockerfiles/scripts:/tmp/scripts
database:
- image: ghcr.io/stac-utils/pgstac:v0.8.1
+ image: ghcr.io/stac-utils/pgstac:v0.8.5
environment:
- POSTGRES_USER=username
- POSTGRES_PASSWORD=password
diff --git a/dockerfiles/Dockerfile.raster b/dockerfiles/Dockerfile.raster
deleted file mode 100644
index 926425c..0000000
--- a/dockerfiles/Dockerfile.raster
+++ /dev/null
@@ -1,14 +0,0 @@
-ARG PYTHON_VERSION=3.11
-
-FROM ghcr.io/vincentsarago/uvicorn-gunicorn:${PYTHON_VERSION}
-
-ENV CURL_CA_BUNDLE /etc/ssl/certs/ca-certificates.crt
-
-RUN pip install psycopg[binary,pool]
-
-COPY runtime/eoapi/raster /tmp/raster
-RUN pip install /tmp/raster
-RUN rm -rf /tmp/raster
-
-ENV MODULE_NAME eoapi.raster.app
-ENV VARIABLE_NAME app
diff --git a/dockerfiles/Dockerfile.raster-uvicorn b/dockerfiles/Dockerfile.raster-uvicorn
deleted file mode 100644
index e5e00c1..0000000
--- a/dockerfiles/Dockerfile.raster-uvicorn
+++ /dev/null
@@ -1,26 +0,0 @@
-ARG PYTHON_VERSION=3.11
-
-FROM bitnami/python:${PYTHON_VERSION}
-RUN apt update && apt upgrade -y \
- && apt install curl -y \
- && rm -rf /var/lib/apt/lists/*
-
-# Ensure root certificates are always updated at evey container build
-# and curl is using the latest version of them
-RUN mkdir /usr/local/share/ca-certificates/cacert.org
-RUN cd /usr/local/share/ca-certificates/cacert.org && curl -k -O https://www.cacert.org/certs/root.crt
-RUN cd /usr/local/share/ca-certificates/cacert.org && curl -k -O https://www.cacert.org/certs/class3.crt
-RUN update-ca-certificates
-ENV CURL_CA_BUNDLE /etc/ssl/certs/ca-certificates.crt
-
-RUN python -m pip install pip -U
-
-RUN python -m pip install psycopg[binary,pool]
-
-COPY runtime/eoapi/raster /tmp/raster
-RUN python -m pip install /tmp/raster uvicorn
-RUN rm -rf /tmp/raster
-
-ENV HOST 0.0.0.0
-ENV PORT 80
-CMD uvicorn eoapi.raster.app:app --host ${HOST} --port ${PORT}
diff --git a/dockerfiles/Dockerfile.stac b/dockerfiles/Dockerfile.stac
deleted file mode 100644
index 287f9be..0000000
--- a/dockerfiles/Dockerfile.stac
+++ /dev/null
@@ -1,12 +0,0 @@
-ARG PYTHON_VERSION=3.11
-
-FROM ghcr.io/vincentsarago/uvicorn-gunicorn:${PYTHON_VERSION}
-
-ENV CURL_CA_BUNDLE /etc/ssl/certs/ca-certificates.crt
-
-COPY runtime/eoapi/stac /tmp/stac
-RUN pip install /tmp/stac
-RUN rm -rf /tmp/stac
-
-ENV MODULE_NAME eoapi.stac.app
-ENV VARIABLE_NAME app
diff --git a/dockerfiles/Dockerfile.stac-uvicorn b/dockerfiles/Dockerfile.stac-uvicorn
deleted file mode 100644
index 0f57d42..0000000
--- a/dockerfiles/Dockerfile.stac-uvicorn
+++ /dev/null
@@ -1,24 +0,0 @@
-ARG PYTHON_VERSION=3.11
-
-FROM bitnami/python:${PYTHON_VERSION}
-RUN apt update && apt upgrade -y \
- && apt install curl -y \
- && rm -rf /var/lib/apt/lists/*
-
-# Ensure root certificates are always updated at evey container build
-# and curl is using the latest version of them
-RUN mkdir /usr/local/share/ca-certificates/cacert.org
-RUN cd /usr/local/share/ca-certificates/cacert.org && curl -k -O https://www.cacert.org/certs/root.crt
-RUN cd /usr/local/share/ca-certificates/cacert.org && curl -k -O https://www.cacert.org/certs/class3.crt
-RUN update-ca-certificates
-ENV CURL_CA_BUNDLE /etc/ssl/certs/ca-certificates.crt
-
-RUN python -m pip install pip -U
-
-COPY runtime/eoapi/stac /tmp/stac
-RUN python -m pip install /tmp/stac uvicorn
-RUN rm -rf /tmp/stac
-
-ENV HOST 0.0.0.0
-ENV PORT 80
-CMD uvicorn eoapi.stac.app:app --host ${HOST} --port ${PORT}
diff --git a/dockerfiles/Dockerfile.vector b/dockerfiles/Dockerfile.vector
deleted file mode 100644
index f49c462..0000000
--- a/dockerfiles/Dockerfile.vector
+++ /dev/null
@@ -1,10 +0,0 @@
-ARG PYTHON_VERSION=3.11
-
-FROM ghcr.io/vincentsarago/uvicorn-gunicorn:${PYTHON_VERSION}
-
-COPY runtime/eoapi/vector /tmp/vector
-RUN pip install /tmp/vector
-RUN rm -rf /tmp/vector
-
-ENV MODULE_NAME eoapi.vector.app
-ENV VARIABLE_NAME app
diff --git a/dockerfiles/Dockerfile.vector-uvicorn b/dockerfiles/Dockerfile.vector-uvicorn
deleted file mode 100644
index 479ad73..0000000
--- a/dockerfiles/Dockerfile.vector-uvicorn
+++ /dev/null
@@ -1,24 +0,0 @@
-ARG PYTHON_VERSION=3.11
-
-FROM bitnami/python:${PYTHON_VERSION}
-RUN apt update && apt upgrade -y \
- && apt install curl -y \
- && rm -rf /var/lib/apt/lists/*
-
-# Ensure root certificates are always updated at evey container build
-# and curl is using the latest version of them
-RUN mkdir /usr/local/share/ca-certificates/cacert.org
-RUN cd /usr/local/share/ca-certificates/cacert.org && curl -k -O https://www.cacert.org/certs/root.crt
-RUN cd /usr/local/share/ca-certificates/cacert.org && curl -k -O https://www.cacert.org/certs/class3.crt
-RUN update-ca-certificates
-ENV CURL_CA_BUNDLE /etc/ssl/certs/ca-certificates.crt
-
-RUN python -m pip install pip -U
-
-COPY runtime/eoapi/vector /tmp/vector
-RUN python -m pip install /tmp/vector uvicorn
-RUN rm -rf /tmp/vector
-
-ENV HOST 0.0.0.0
-ENV PORT 80
-CMD uvicorn eoapi.vector.app:app --host ${HOST} --port ${PORT}
diff --git a/docs/src/contributing.md b/docs/src/contributing.md
deleted file mode 100644
index 1300390..0000000
--- a/docs/src/contributing.md
+++ /dev/null
@@ -1,42 +0,0 @@
-Issues and pull requests are more than welcome: https://github.com/developmentseed/eoAPI/issues
-
-You can also start **Discussions** in https://github.com/developmentseed/eoAPI/discussions
-
-**dev install**
-
-```bash
-# Download the code
-git clone https://github.com/developmentseed/eoAPI.git
-cd eoAPI
-
-# Create a virtual environment
-python -m pip install --upgrade virtualenv
-virtualenv .venv
-source .venv/bin/activate
-
-# Install the eoapi module
-python -m pip install "psycopg[binary,pool]"
-python -m pip install -e runtime/eoapi/raster["test"] # or -e runtime/eoapi/stac["test"] | -e runtime/eoapi/vector["test"]
-```
-
-!!! danger
-
- Python applications might have incompatible dependencies, which you can resolve by using a virtual environment *per application*
-
-**pre-commit**
-
-This repo is set to use `pre-commit` to run *isort*, *ruff*, *pydocstring*, *black* ("uncompromising Python code formatter") and mypy when committing new code.
-
-```bash
-$ pre-commit install
-```
-
-### Open Source
-
-You can also contribute indirectly to eoAPI by helping on the sub-modules:
-
-- **PgSTAC** database https://github.com/stac-utils/pgstac
-- **stac-fastapi**: https://github.com/stac-utils/stac-fastapi
-- **titiler-pgstac**: https://github.com/stac-utils/titiler-pgstac
-- **TiTiler**: https://github.com/developmentseed/titiler
-- **TiPg**: https://github.com/developmentseed/tipg
\ No newline at end of file
diff --git a/docs/src/contributing.md b/docs/src/contributing.md
new file mode 120000
index 0000000..f939e75
--- /dev/null
+++ b/docs/src/contributing.md
@@ -0,0 +1 @@
+../../CONTRIBUTING.md
\ No newline at end of file
diff --git a/docs/src/customization.md b/docs/src/customization.md
index 01dad40..dce2b80 100644
--- a/docs/src/customization.md
+++ b/docs/src/customization.md
@@ -1,71 +1,70 @@
-The eoAPI repository (https://github.com/developmentseed/eoAPI) hosts customized versions of each base service. The documentation below demonstrates how each service can be customized. The eoAPI services can work in parallel or in combination with each other.
----
+The **eoapi-devseed** [repository](https://github.com/developmentseed/eoapi-devseed) hosts customized versions of each base service. The documentation below demonstrates how each service can be customized. The eoAPI services can work in parallel or combination with each other.
+
## eoapi.stac
Built on [stac-fastapi.pgstac](https://github.com/stac-utils/stac-fastapi-pgstac) application, adding a **`TiTilerExtension`** and a simple **`Search Viewer`**.
The service includes:
-- Full **stac-fastapi** implementation - see [docs](http://localhost:8081/docs) if using the `docker-compose` configuration.
-- Simple STAC Search **viewer** - see [viewer](http://localhost:8081/index.html) if using the `docker-compose` configuration.
+- If you use the `docker-compose` configuration, see [docs](http://localhost:8081/docs) for the full `stac-fastapi` implementation.
+- Simple STAC Search viewer—see [viewer](http://localhost:8081/index.html) if you are using the `docker-compose` configuration.
- **Proxy** to the tiler endpoint for STAC Items.
-When the `TITILER_ENDPOINT` environment variable is set (pointing to the `raster` application), additional endpoints will be added to the stac-fastapi application (see: [stac/extension.py](https://github.com/developmentseed/eoAPI/blob/main/src/eoapi/stac/eoapi/stac/extension.py)):
+When the `TITILER_ENDPOINT` environment variable is set (pointing to the `raster` application), additional endpoints will be added to the stac-fastapi application (see: [stac/extension.py](https://github.com/developmentseed/eoapi-devseed/blob/main/runtimes/eoapi/stac/eoapi/stac/extension.py)):
- `/collections/{collectionId}/items/{itemId}/tilejson.json`: Return the `raster` tilejson for an item
- `/collections/{collectionId}/items/{itemId}/viewer`: Redirect to the `raster` viewer
-
-
+
+
-Code: [/runtime/eoapi/stac](https://github.com/developmentseed/eoAPI/tree/main/runtime/eoapi/stac)
+Code: [/runtimes/eoapi/stac](https://github.com/developmentseed/eoapi-devseed/tree/main/runtimes/eoapi/stac)
---
+
## eoapi.raster
-The dynamic tiler deployed within `eoAPI` is built on top of [titiler-pgstac](https://github.com/stac-utils/titiler-pgstac) and [pgstac](https://github.com/stac-utils/pgstac). It enables large-scale mosaic based on the results of STAC search queries.
+The dynamic tiler deployed within `eoapi-devseed` is built on top of [titiler-pgstac](https://github.com/stac-utils/titiler-pgstac) and [pgstac](https://github.com/stac-utils/pgstac). It enables large-scale mosaic based on the results of STAC search queries.
The service includes all the default endpoints from **titiler-pgstac** application and:
- `/`: a custom landing page with links to the different endpoints
-
-- `/mosaic/builder`: a virtual mosaic builder UI, which helps create and register STAC Search queries
-
-- `/collections`: a *secret* (not in OpenAPI documentation) endpoint used in the mosaic-builder page
-
+- `/mosaic/builder`: a virtual mosaic builder UI that helps create and register STAC Search queries
+- `/collections`: a secret (not in OpenAPI documentation) endpoint used in the mosaic-builder page
- `/collections/{collection_id}/items/{item_id}/viewer`: a simple STAC Item viewer
-
-
-
+
+
+
-
-Code: [/runtime/eoapi/raster](https://github.com/developmentseed/eoAPI/tree/main/runtime/eoapi/raster)
+Code: [/runtimes/eoapi/raster](https://github.com/developmentseed/eoapi-devseed/tree/main/runtimes/eoapi/raster)
---
+
## eoapi.vector
OGC Features and Tiles API built on top of [tipg](https://github.com/developmentseed/tipg).
-By default, the API will look for tables in the `public` schema of the database. We've also added three functions that connect to the pgSTAC schema:
+The API will look for tables in the database's `public` schema by default. We've also added three functions that connect to the pgSTAC schema:
- **pg_temp.pgstac_collections_view**: Simple function which returns PgSTAC Collections
- **pg_temp.pgstac_hash**: Return features for a specific `searchId` (hash)
- **pg_temp.pgstac_hash_count**: Return the number of items per geometry for a specific `searchId` (hash)
-
-
+
+
-Code: [/runtime/eoapi/vector](https://github.com/developmentseed/eoAPI/tree/main/runtime/eoapi/vector)
+Code: [/runtimes/eoapi/vector](https://github.com/developmentseed/eoapi-devseed/tree/main/runtimes/eoapi/vector)
+
+---
---
# STAC browser
-The custom browser configuration can be modified using the config located in [/dockerfiles/browser_config.js](https://github.com/developmentseed/eoAPI/tree/main/dockerfiles/browser_config.js). For more information about available configurations, see the [Radiant Earth repository](https://github.com/radiantearth/stac-browser).
+The custom browser configuration can be modified using the config located in [/dockerfiles/browser_config.js](https://github.com/developmentseed/eoapi-devseed/blob/main/dockerfiles/browser_config.js). For more information about available configurations, see the [Radiant Earth repository](https://github.com/radiantearth/stac-browser).
diff --git a/docs/src/deployment.md b/docs/src/deployment.md
index cdeb8a0..4860277 100644
--- a/docs/src/deployment.md
+++ b/docs/src/deployment.md
@@ -1,13 +1,11 @@
----
+
hide:
- - navigation
+- navigation
---
-
## Via [eoapi-cdk](https://github.com/developmentseed/eoapi-cdk)
-
-[eoapi-cdk](https://github.com/developmentseed/eoapi-cdk) is a set of AWS CDK constructs that can be used to easily deploy eoAPI services on AWS with the CDK.
+[eoapi-cdk](https://github.com/developmentseed/eoapi-cdk) is a set of AWS CDK constructs that can readily deploy eoAPI services on AWS with the CDK.
[eoapi-template](https://github.com/developmentseed/eoapi-template) is an AWS CDK app that shows how to configure the [eoapi-cdk](https://github.com/developmentseed/eoapi-cdk) constructs.
@@ -18,146 +16,140 @@ The stack is deployed by the [AWS CDK](https://aws.amazon.com/cdk/) utility. Und
The example commands here will deploy a CloudFormation stack called `eoAPI-staging`.
1. Clone the `eoapi` repo and install dependencies
- ```bash
- # Download eoapi repo
- git clone https://github.com/developmentseed/eoapi.git
-
- # Create a virtual environment
- python -m pip install --upgrade virtualenv
- virtualenv infrastructure/aws/.venv
- source infrastructure/aws/.venv/bin/activate
- # install cdk dependencies
- python -m pip install -r infrastructure/aws/requirements-cdk.txt
- ```
+```bash
+# Download eoapi repo
+git clone https://github.com/developmentseed/eoapi-template.git
+cd eoapi-template
-2. Install node dependency - requires node version 14+
- ```bash
- npm --prefix infrastructure/aws install
- ```
+# Create a virtual environment
+python -m venv .venv
+source .venv/bin/activate
-3. Update settings
+# install cdk dependencies
+python -m pip install -r requirements.txt
+```
- Set environment variable or complex code in the `infrastructure/aws/.env` file (e.g., `CDK_EOAPI_DB_PGSTAC_VERSION=0.7.1`).
+2. Install node dependency - requires node version 14+
- To modify the size of the burstable database instance, modify `CDK_EOAPI_DB_INSTANCE_SIZE` to one of the values of [`aws_cdk.aws_ec2.InstanceSize`](https://docs.aws.amazon.com/cdk/api/v2/python/aws_cdk.aws_ec2/InstanceSize.html#instancesize).
- The default size is `SMALL`.
+```bash
+npm install
+```
- **Important**:
+3. Update settings
- - `CDK_EOAPI_DB_PGSTAC_VERSION` is a required env (see https://github.com/stac-utils/pgstac/tags for the latest version)
+Set environment variable or complex code in the `.env` or `config.yaml` file (e.g., https://github.com/developmentseed/eoapi-template/blob/main/config.yaml.example).
- - You can choose which functions to deploy by setting `CDK_EOAPI_FUNCTIONS` env (e.g., `CDK_EOAPI_FUNCTIONS='["stac","raster","vector"]'`)
+See https://github.com/developmentseed/eoapi-template/blob/main/infrastructure/config.py for more info on the configuration options.
+4. Install CDK and connect to your AWS account. This step is only necessary once per AWS account. The environment variables `PROJECT_ID` and `STAGE` determine the name of the stack (e.g., eoAPI-staging or eoAPI-production)
-4. Install CDK and connect to your AWS account. This step is only necessary once per AWS account. The environment variable `CDK_EOAPI_STAGE` determines the name of the stack
-(e.g., eoAPI-staging or eoAPI-production)
- ```bash
- # Deploy the CDK toolkit stack into an AWS environment.
- CDK_EOAPI_STAGE=staging \
- CDK_EOAPI_DB_PGSTAC_VERSION=0.7.1 \
- npm --prefix infrastructure/aws run cdk -- bootstrap
+```bash
+# Deploy the CDK toolkit stack into an AWS environment.
+PROJECT_ID=eoAPI \
+STAGE=staging \
+npx cdk bootstrap
- # or to a specific region
- AWS_DEFAULT_REGION=us-west-2 \
- AWS_REGION=us-west-2 \
- CDK_EOAPI_STAGE=staging \
- CDK_EOAPI_DB_PGSTAC_VERSION=0.7.1 \
- npm --prefix infrastructure/aws run cdk -- bootstrap
- ```
+# or to a specific region
+AWS_DEFAULT_REGION=us-west-2 \
+AWS_REGION=us-west-2 \
+PROJECT_ID=eoAPI \
+STAGE=staging \
+npx cdk bootstrap
+```
5. Pre-Generate CFN template
- ```bash
- CDK_EOAPI_STAGE=staging \
- CDK_EOAPI_DB_PGSTAC_VERSION=0.7.1 \
- npm --prefix infrastructure/aws run cdk -- synth # Synthesizes and prints the CloudFormation template for this stack
- ```
+```bash
+PROJECT_ID=eoAPI \
+STAGE=staging \
+npx cdk synth --all # Synthesizes and prints the CloudFormation template for this stack
+```
6. Deploy
- ```bash
- CDK_EOAPI_STAGE=staging \
- CDK_EOAPI_DB_PGSTAC_VERSION=0.7.1 \
- npm --prefix infrastructure/aws run cdk -- deploy eoAPI-staging
+```bash
+# Note: a VPC stack is needed for the database
+PROJECT_ID=eoAPI \
+STAGE=staging \
+npx cdk deploy vpceoAPI-staging eoAPI-staging
- # Deploy in a specific region
- AWS_DEFAULT_REGION=eu-central-1 \
- AWS_REGION=eu-central-1 \
- CDK_EOAPI_STAGE=staging \
- CDK_EOAPI_DB_PGSTAC_VERSION=0.7.1 \
- npm --prefix infrastructure/aws run cdk -- deploy eoapi-staging --profile {my-aws-profile}
- ```
+# Deploy in a specific region
+AWS_DEFAULT_REGION=eu-central-1 \
+AWS_REGION=eu-central-1 \
+PROJECT_ID=eoAPI \
+STAGE=staging \
+npx cdk deploy vpceoAPI-staging eoAPI-staging --profile {my-aws-profile}
+```
-If you get an error saying that the max VPCs have been reached, you have hit the limit for the number of VPCs per unique AWS account and region combination. You can change the AWS region to a region with fewer VPCs and deploy again to fix this.
+If you get an error saying that the max VPCs have been reached, you have hit the limit for the number of VPCs per unique AWS account and region combination. To fix this, you can change the AWS region to a region with fewer VPCs and deploy again.
## Via [eoapi-k8s](https://github.com/developmentseed/eoapi-k8s)
[eoapi-k8s](https://github.com/developmentseed/eoapi-k8s) has the IaC and Helm charts for deploying eoAPI services on AWS and GCP.
-**Getting started**
+### Getting started
-If you still need to set up a k8s cluster on AWS or GCP, then follow an IaC guide below that is relevant to you.
+If you still need to set up a K8s cluster on AWS or GCP, follow the relevant IaC guide below.
> ⓘ The helm chart in this repo assumes your cluster has a few third-party add-ons and controllers installed. So
> It's in your best interest to read through the IaC guides to understand what those defaults are
-* :octicons-link-external-16: [AWS EKS Cluster Setup](https://github.com/developmentseed/eoapi-k8s/blob/main/docs/aws-eks.md)
-
-* :octicons-link-external-16: [TBD: GCP GKE Cluster Setup](https://github.com/developmentseed/eoapi-k8s/blob/main/docs/gcp-gke.md)
+- :octicons-link-external-16: [AWS EKS Cluster Setup](https://github.com/developmentseed/eoapi-k8s/blob/main/docs/aws-eks.md)
+- :octicons-link-external-16: [TBD: GCP GKE Cluster Setup](https://github.com/developmentseed/eoapi-k8s/blob/main/docs/gcp-gke.md)
-**Helm Installation**
+### Helm Installation
-Once you have a k8s cluster set up, you can `helm install` eoAPI as follows
+Once you have a k8s cluster, you can `helm install` eoAPI as follows.
1. `helm install` from this repo's `helm-chart/` folder:
- ```python
- ######################################################
- # create os environment variables for required secrets
- ######################################################
- $ export GITSHA=$(git rev-parse HEAD | cut -c1-10)
- $ export PGUSER=s00pers3cr3t
- $ export POSTGRES_USER=s00pers3cr3t
- $ export POSTGRES_PASSWORD=superuserfoobar
- $ export PGPASSWORD=foobar
-
- $ cd ./helm-chart
-
- $ helm install \
- --namespace eoapi \
- --create-namespace \
- --set gitSha=$GITSHA \
- --set db.settings.secrets.PGUSER=$PGUSER \
- --set db.settings.secrets.POSTGRES_USER=$POSTGRES_USER \
- --set db.settings.secrets.PGPASSWORD=$PGPASSWORD \
- --set db.settings.secrets.POSTGRES_PASSWORD=$POSTGRES_PASSWORD \
- eoapi \
- ./eoapi
- ```
+```bash
+######################################################
+# create os environment variables for required secrets
+######################################################
+export GITSHA=$(git rev-parse HEAD | cut -c1-10)
+export PGUSER=s00pers3cr3t
+export POSTGRES_USER=s00pers3cr3t
+export POSTGRES_PASSWORD=superuserfoobar
+export PGPASSWORD=foobar
+
+cd ./helm-chart
+helm install \
+--namespace eoapi \
+--create-namespace \
+--set gitSha=$GITSHA \
+--set db.settings.secrets.PGUSER=$PGUSER \
+--set db.settings.secrets.POSTGRES_USER=$POSTGRES_USER \
+--set db.settings.secrets.PGPASSWORD=$PGPASSWORD \
+--set db.settings.secrets.POSTGRES_PASSWORD=$POSTGRES_PASSWORD \
+eoapi \
+./eoapi
+```
2. or `helm install` from https://devseed.com/eoapi-k8s/:
- ```python
- # add the eoapi helm repo locally
- $ helm repo add eoapi https://devseed.com/eoapi-k8s/
-
- # List out the eoapi chart versions
- $ helm search repo eoapi
- NAME CHART VERSION APP VERSION DESCRIPTION
- eoapi/eoapi 0.1.1 0.1.0 Create a full Earth Observation API with Metada...
- eoapi/eoapi 0.1.2 0.1.0 Create a full Earth Observation API with Metada...
-
- # add the required secret overrides to an arbitrarily named `.yaml` file (`config.yaml` below)
- $ cat config.yaml
- db:
- settings:
- secrets:
- PGUSER: "username"
- POSTGRES_USER: "username"
- PGPASSWORD: "password"
- POSTGRES_PASSWORD: "password"
-
- # then run `helm install` with those overrides
- helm install eoapi eoapi/eoapi --version 0.1.1 -f config.yaml
- ```
\ No newline at end of file
+```bash
+# add the eoapi helm repo locally
+helm repo add eoapi https://devseed.com/eoapi-k8s/
+
+# List out the eoapi chart versions
+helm search repo eoapi
+
+NAME CHART VERSION APP VERSION DESCRIPTION
+eoapi/eoapi 0.1.1 0.1.0 Create a full Earth Observation API with Metada...
+eoapi/eoapi 0.1.2 0.1.0 Create a full Earth Observation API with Metada...
+
+# add the required secret overrides to an arbitrarily named `.yaml` file (`config.yaml` below)
+cat config.yaml
+db:
+ settings:
+ secrets:
+ PGUSER: "username"
+ POSTGRES_USER: "username"
+ PGPASSWORD: "password"
+ POSTGRES_PASSWORD: "password"
+
+# then run `helm install` with those overrides
+helm install eoapi eoapi/eoapi --version 0.1.1 -f config.yaml
+```
diff --git a/docs/src/intro.md b/docs/src/intro.md
index ffe4789..2c187c3 100644
--- a/docs/src/intro.md
+++ b/docs/src/intro.md
@@ -1,24 +1,20 @@
+
Create a full Earth Observation API with Metadata, Raster, and Vector services.
-
---
-## **E**arth **O**bservation **API**
-
-`eoAPI` combines several *state-of-the-art* projects to create a full Earth Observation API. Each service can be used and deployed independently, but `eoAPI` creates the interconnections between each service:
-
-- **pgSTAC** database [https://github.com/stac-utils/pgstac](https://github.com/stac-utils/pgstac)
-
-- **STAC API** built on top of [https://github.com/stac-utils/stac-fastapi](https://github.com/stac-utils/stac-fastapi)
-
-- **STAC browser** a UI that exposes, in a user friendly interface, the metadata served by the STAC API. Built on top of [https://github.com/radiantearth/stac-browser](https://github.com/radiantearth/stac-browser)
+## Earth Observation API
-- **STAC Items And Mosaic Raster Tiles** API built on top of [https://github.com/stac-utils/titiler-pgstac](https://github.com/stac-utils/titiler-pgstac)
+`eoAPI` combines several **state-of-the-art** projects to create a full Earth Observation API. Each service can be used and deployed independently, but `eoAPI` creates the interconnections between each service:
-- **OGC Features and Vector Tiles** API built on top of [https://github.com/developmentseed/tipg](https://github.com/developmentseed/tipg)
+- **pgSTAC** database https://github.com/stac-utils/pgstac
+- **STAC API** built on top of https://github.com/stac-utils/stac-fastapi
+- **STAC browser** a UI that exposes, in a user-friendly interface, the metadata served by the STAC API. Built on top of https://github.com/radiantearth/stac-browser
+- **STAC Items And Mosaic Raster Tiles** API built on top of https://github.com/stac-utils/titiler-pgstac
+- **OGC Features and Vector Tiles** API built on top of https://github.com/developmentseed/tipg
---
@@ -30,58 +26,47 @@
## Why should you use `eoAPI`
-- **Focus on your use case:**Â `eoAPI`Â is used for large-scale data processing, building geographic information systems (GIS), creating real-time data applications, climate research and environmental monitoring, machine learning model training, and more.
-
-- **Unified Repository:**Â `eoAPI`Â provides a single, unified repository for several state-of-the-art Earth Observation (EO) data services, including Metadata search (STAC), Raster, and Vector services. This can simplify the process of accessing and working with these services.
-
-- **Interoperability:**Â `eoAPI`Â is designed to enable interoperability among its included services. This can make building complex applications that leverage different types of EO data easier.
-
-- **Open Source and Community Support:** As an open-source project, `eoAPI` allows developers to inspect its code, contribute to its development, and use it as a base for custom solutions. It also benefits from the support and innovation of a community of developers and EO data users.
-
-- **Scalability and Flexibility:** Each service in `eoAPI` can be used or deployed independently, which provides a lot of flexibility. If a developer's application only requires one or two of eoAPI's services, they don't need to deploy the entire suite.
-
-- **Facilitate Earth Observation Tasks:**Â `eoAPI`Â includes specialized tools for working with EO data, such as dynamic tiling, metadata searching, and features/vector tiles API. These can significantly facilitate EO data processing, analysis, and visualization.
-
-- **Ease of Deployment:**Â `eoAPI`Â supports containerized deployment using Docker, making it easier to set up, scale, and maintain applications built on it. Spin up the demo locally and start experimenting in minutes.
+- **Focus on your use case**: `eoAPI` is used for large-scale data processing, building geographic information systems (GIS), creating real-time data applications, climate research and environmental monitoring, machine learning model training, and more.
+- **Unified Repository**: `eoAPI` provides a single, unified repository for several state-of-the-art Earth Observation (EO) data services, including Metadata search (STAC), Raster, and Vector services. This can simplify accessing and working with these services.
+- **Interoperability**: `eoAPI` is designed to enable interoperability among its included services. This can make building complex applications that leverage different types of EO data easier.
+- **Open Source and Community Support**: As an open-source project, `eoAPI` allows developers to inspect its code, contribute to its development, and use it as a base for custom solutions. It also benefits from the support and innovation of a community of developers and EO data users.
+- **Scalability and Flexibility**: Each service in `eoAPI` can be used or deployed independently, which provides a lot of flexibility. If a developer's application only requires one or two of eoAPI's services, they don't need to deploy the entire suite.
+- **Facilitate Earth Observation Tasks**: `eoAPI` includes specialized tools for working with EO data, such as dynamic tiling, metadata searching, and features/vector tiles API. These can significantly facilitate EO data processing, analysis, and visualization.
+- **Ease of Deployment**: `eoAPI` supports containerized deployment using Docker, making setting up, scaling, and maintaining applications built on it more accessible. Spin up the demo locally and start experimenting in minutes.
---
## Services Overview
- **STAC Metadata**: Built with [stac-fastapi.pgstac](https://github.com/stac-utils/stac-fastapi) to enable data discovery. See the specifications [core](https://github.com/radiantearth/stac-api-spec/blob/v1.0.0/core/README.md), [search](https://github.com/radiantearth/stac-api-spec/blob/v1.0.0/item-search/README.md) and [features](https://github.com/radiantearth/stac-api-spec/blob/v1.0.0/ogcapi-features/README.md) for API details.
-
-- **STAC browser** : Built with the [Radiant Earth STAC browser](https://github.com/radiantearth/stac-browser) to provide a simple user-friendly interface for searching the STAC metadata.
-
-- **Raster Tiles**: Built with [titiler-pgstac](https://github.com/stac-utils/titiler-pgstac) and [pgstac](https://github.com/stac-utils/pgstac) to enable large scale mosaic based on results of STAC searches queries. See [docs](https://stac-utils.github.io/titiler-pgstac/0.8.0/mosaic_endpoints/) for API details.
-
+- **STAC browser**: Built with the [Radiant Earth STAC browser](https://github.com/radiantearth/stac-browser) to provide a simple, user-friendly interface for searching the STAC metadata.
+- **Raster Tiles**: Built with [titiler-pgstac](https://github.com/stac-utils/titiler-pgstac) and [pgstac](https://github.com/stac-utils/pgstac) to enable large-scale mosaic based on results of STAC searches queries. See [docs](https://stac-utils.github.io/titiler-pgstac/0.8.0/mosaic_endpoints/) for API details.
- **OGC Features & Vector Tiles**: Built with [tipg](https://github.com/developmentseed/tipg) to create a lightweight OGC Features and Tiles API with a PostGIS database. See [docs](https://developmentseed.org/tipg/user_guide/endpoints/) for API details.
-
See [service details](./services.md) for more information.
---
## Getting started
-The easiest way to start exploring the different eoAPI services is with *Docker*. Clone this repository and start the multi-container *Docker* applications using `Compose`:
+The easiest way to explore the different eoAPI services is with **Docker**. Clone this repository and start the multi-container **Docker** applications using `Compose`:
-```
+```bash
git clone https://github.com/developmentseed/eoAPI.git
cd eoAPI
docker compose up
```
-Once the applications are *up*, you'll need to add STAC **Collections** and **Items** to the PgSTAC database. If you don't have, you can use the follow the [MAXAR open data demo](https://github.com/vincentsarago/MAXAR_opendata_to_pgstac) (or get inspired by the other [demos](https://github.com/developmentseed/eoAPI/tree/main/demo)).
-
+Once the applications are **up**, you'll need to add STAC **Collections** and **Items** to the PgSTAC database. If you don't have a STAC collection, you can use the [MAXAR open data demo](https://github.com/vincentsarago/MAXAR_opendata_to_pgstac) (or get inspired by the other [demos](https://github.com/developmentseed/eoAPI/tree/main/demo)).
Then you can start exploring your dataset with:
- - the STAC Metadata service [http://localhost:8081](http://localhost:8081)
- - the Raster service [http://localhost:8082](http://localhost:8082)
+- the STAC Metadata service http://localhost:8081
+- the Raster service http://localhost:8082
!!! info
- If you've added vector datasets to the `public` schema in the Postgres database, they will be available through the **Vector** service at [http://localhost:8083](http://localhost:8083).
+If you've added vector datasets to the `public` schema in the Postgres database, they will be available through the **Vector** service at http://localhost:8083.
Alternatively, you may install and launch applications locally:
@@ -91,29 +76,28 @@ virtualenv .venv
source .venv/bin/activate
export DATABASE_URL=postgresql://username:password@0.0.0.0:5439/postgis # Connect to the database of your choice
-
python -m pip install uvicorn
###############################################################################
# Install and launch the application
# Select one of the following
-
###############################################################################
+
# STAC
python -m pip install "psycopg[binary,pool]" stac-fastapi-pgstac
-.venv/bin/uvicorn stac_fastapi.pgstac.app:app --port 8081 --reload
+python -m uvicorn stac_fastapi.pgstac.app:app --port 8081 --reload
###############################################################################
# RASTER
python -m pip install "psycopg[binary,pool]" titiler-pgstac
-.venv/bin/uvicorn titiler.pgstac.main:app --port 8082 --reload
+python -m uvicorn titiler.pgstac.main:app --port 8082 --reload
###############################################################################
# VECTOR
python -m pip install tipg
-.venv/bin/uvicorn tipg.main:app --port 8083 --reload
+python -m uvicorn tipg.main:app --port 8083 --reload
```
!!! danger
- Python applications might have incompatible dependencies, which you can resolve by using a virtual environment *per application*
+Python applications might have incompatible dependencies, which you can resolve by using a virtual environment *per application*
diff --git a/infrastructure/aws/.env.example b/infrastructure/aws/.env.example
deleted file mode 100644
index 5c27dbb..0000000
--- a/infrastructure/aws/.env.example
+++ /dev/null
@@ -1,19 +0,0 @@
-# STACK
-CDK_EOAPI_NAME=eoAPI
-CDK_EOAPI_FUNCTIONS='["stac","raster","vector"]'
-
-# DB
-CDK_EOAPI_DB_PGSTAC_VERSION="0.8.1"
-CDK_EOAPI_DB_PGSTAC_MOSAIC_INDEX=TRUE
-
-# STAC API
-CDK_EOAPI_STAC_TIMEOUT=20
-CDK_EOAPI_STAC_MEMORY=1024
-
-# Raster API
-CDK_EOAPI_RASTER_TIMEOUT=20
-CDK_EOAPI_RASTER_MEMORY=3008
-
-# Vector
-CDK_EOAPI_VECTOR_TIMEOUT=20
-CDK_EOAPI_VECTOR_MEMORY=1024
diff --git a/infrastructure/aws/browser_config.example.js b/infrastructure/aws/browser_config.example.js
deleted file mode 100644
index 6c8f9de..0000000
--- a/infrastructure/aws/browser_config.example.js
+++ /dev/null
@@ -1,39 +0,0 @@
-module.exports = {
- catalogUrl: null,
- catalogTitle: "eoAPI STAC Browser",
- allowExternalAccess: true, // Must be true if catalogUrl is not given
- allowedDomains: [],
- detectLocaleFromBrowser: true,
- storeLocale: true,
- locale: "en",
- fallbackLocale: "en",
- supportedLocales: [
- "de",
- "es",
- "en",
- "fr",
- "it",
- "ro"
- ],
- apiCatalogPriority: null,
- useTileLayerAsFallback: true,
- displayGeoTiffByDefault: false,
- buildTileUrlTemplate: ({href, asset}) => "https://raster.dev/cog/tiles/{z}/{x}/{y}@2x?url=" + encodeURIComponent(asset.href.startsWith("/vsi") ? asset.href : href),
- stacProxyUrl: null,
- pathPrefix: "/",
- historyMode: "history",
- cardViewMode: "cards",
- cardViewSort: "asc",
- showThumbnailsAsAssets: false,
- stacLint: true,
- geoTiffResolution: 128,
- redirectLegacyUrls: false,
- itemsPerPage: 12,
- defaultThumbnailSize: null,
- maxPreviewsOnMap: 50,
- crossOriginMedia: null,
- requestHeaders: {},
- requestQueryParameters: {},
- preprocessSTAC: null,
- authConfig: null
-};
\ No newline at end of file
diff --git a/infrastructure/aws/cdk.json b/infrastructure/aws/cdk.json
deleted file mode 100644
index 57cc60c..0000000
--- a/infrastructure/aws/cdk.json
+++ /dev/null
@@ -1,3 +0,0 @@
-{
- "app": "python3 cdk/app.py"
-}
diff --git a/infrastructure/aws/cdk/__init__.py b/infrastructure/aws/cdk/__init__.py
deleted file mode 100644
index 4955682..0000000
--- a/infrastructure/aws/cdk/__init__.py
+++ /dev/null
@@ -1 +0,0 @@
-"""AWS App."""
diff --git a/infrastructure/aws/cdk/app.py b/infrastructure/aws/cdk/app.py
deleted file mode 100644
index cf14874..0000000
--- a/infrastructure/aws/cdk/app.py
+++ /dev/null
@@ -1,423 +0,0 @@
-"""
-CDK Stack definition code for EOAPI
-"""
-import os
-from typing import Any
-
-import boto3
-from aws_cdk import App, CfnOutput, Duration, RemovalPolicy, Stack, Tags
-from aws_cdk import aws_ec2 as ec2
-from aws_cdk import aws_iam as iam
-from aws_cdk import aws_lambda
-from aws_cdk import aws_logs as logs
-from aws_cdk import aws_rds as rds
-from aws_cdk import aws_s3 as s3
-from config import (
- eoAPISettings,
- eoDBSettings,
- eoRasterSettings,
- eoStacBrowserSettings,
- eoSTACSettings,
- eoVectorSettings,
-)
-from constructs import Construct
-from eoapi_cdk import (
- PgStacApiLambda,
- PgStacDatabase,
- StacBrowser,
- StacIngestor,
- TiPgApiLambda,
- TitilerPgstacApiLambda,
-)
-
-eoapi_settings = eoAPISettings()
-
-
-class eoAPIconstruct(Stack):
- """Earth Observation API CDK application"""
-
- def __init__( # noqa: C901
- self,
- scope: Construct,
- id: str,
- stage: str,
- name: str,
- context_dir: str = "../../",
- **kwargs: Any,
- ) -> None:
- """Define stack."""
- super().__init__(scope, id, **kwargs)
-
- vpc = ec2.Vpc(
- self,
- f"{id}-vpc",
- subnet_configuration=[
- ec2.SubnetConfiguration(
- name="ingress",
- cidr_mask=24,
- subnet_type=ec2.SubnetType.PUBLIC,
- )
- ],
- )
-
- interface_endpoints = [
- (
- "SecretsManager Endpoint",
- ec2.InterfaceVpcEndpointAwsService.SECRETS_MANAGER,
- ),
- (
- "CloudWatch Logs Endpoint",
- ec2.InterfaceVpcEndpointAwsService.CLOUDWATCH_LOGS,
- ),
- ]
- for (key, service) in interface_endpoints:
- vpc.add_interface_endpoint(key, service=service)
-
- gateway_endpoints = [("S3", ec2.GatewayVpcEndpointAwsService.S3)]
- for (key, service) in gateway_endpoints:
- vpc.add_gateway_endpoint(key, service=service)
-
- eodb_settings = eoDBSettings()
-
- pgstac_db = PgStacDatabase(
- self,
- "pgstac-db",
- vpc=vpc,
- engine=rds.DatabaseInstanceEngine.postgres(
- version=rds.PostgresEngineVersion.VER_14
- ),
- vpc_subnets=ec2.SubnetSelection(subnet_type=ec2.SubnetType.PUBLIC),
- allocated_storage=eodb_settings.allocated_storage,
- instance_type=ec2.InstanceType.of(
- ec2.InstanceClass.BURSTABLE3,
- ec2.InstanceSize(eodb_settings.instance_size),
- ),
- database_name="postgres",
- backup_retention=Duration.days(7),
- deletion_protection=eoapi_settings.stage.lower() == "production",
- removal_policy=RemovalPolicy.SNAPSHOT
- if eoapi_settings.stage.lower() == "production"
- else RemovalPolicy.DESTROY,
- custom_resource_properties={
- "pgstac_version": eodb_settings.pgstac_version,
- "context": eodb_settings.context,
- "mosaic_index": eodb_settings.mosaic_index,
- },
- bootstrapper_lambda_function_options={
- "handler": "handler.handler",
- "runtime": aws_lambda.Runtime.PYTHON_3_10,
- "code": aws_lambda.Code.from_docker_build(
- path=os.path.abspath(context_dir),
- file="infrastructure/aws/dockerfiles/Dockerfile.db",
- build_args={
- "PYTHON_VERSION": "3.10",
- "PGSTAC_VERSION": eodb_settings.pgstac_version,
- },
- platform="linux/amd64",
- ),
- "timeout": Duration.minutes(5),
- "allow_public_subnet": True,
- "log_retention": logs.RetentionDays.ONE_WEEK,
- },
- pgstac_db_name=eodb_settings.dbname,
- pgstac_username=eodb_settings.user,
- secrets_prefix=os.path.join(stage, name),
- )
-
- # allow connections from anywhere to the DB
- pgstac_db.db.connections.allow_default_port_from_any_ipv4()
-
- CfnOutput(
- self,
- f"{id}-database-secret-arn",
- value=pgstac_db.pgstac_secret.secret_arn,
- description="Arn of the SecretsManager instance holding the connection info for Postgres DB",
- )
-
- # eoapi.raster
- if "raster" in eoapi_settings.functions:
-
- db_secrets = {
- "POSTGRES_HOST": pgstac_db.pgstac_secret.secret_value_from_json(
- "host"
- ).to_string(),
- "POSTGRES_DBNAME": pgstac_db.pgstac_secret.secret_value_from_json(
- "dbname"
- ).to_string(),
- "POSTGRES_USER": pgstac_db.pgstac_secret.secret_value_from_json(
- "username"
- ).to_string(),
- "POSTGRES_PASS": pgstac_db.pgstac_secret.secret_value_from_json(
- "password"
- ).to_string(),
- "POSTGRES_PORT": pgstac_db.pgstac_secret.secret_value_from_json(
- "port"
- ).to_string(),
- }
-
- eoraster_settings = eoRasterSettings()
- env = eoraster_settings.env or {}
- if "DB_MAX_CONN_SIZE" not in env:
- env["DB_MAX_CONN_SIZE"] = "1"
- env.update(db_secrets)
-
- eoraster = TitilerPgstacApiLambda(
- self,
- f"{id}-raster-lambda",
- db=pgstac_db.db,
- db_secret=pgstac_db.pgstac_secret,
- api_env=env,
- lambda_function_options={
- "code": aws_lambda.Code.from_docker_build(
- path=os.path.abspath(context_dir),
- file="infrastructure/aws/dockerfiles/Dockerfile.raster",
- build_args={
- "PYTHON_VERSION": "3.11",
- },
- platform="linux/amd64",
- ),
- "allow_public_subnet": True,
- "handler": "handler.handler",
- "runtime": aws_lambda.Runtime.PYTHON_3_11,
- "memory_size": eoraster_settings.memory,
- "timeout": Duration.seconds(eoraster_settings.timeout),
- "log_retention": logs.RetentionDays.ONE_WEEK,
- },
- buckets=eoraster_settings.buckets,
- )
-
- # eoapi.stac
- if "stac" in eoapi_settings.functions:
- db_secrets = {
- "POSTGRES_HOST_READER": pgstac_db.pgstac_secret.secret_value_from_json(
- "host"
- ).to_string(),
- "POSTGRES_HOST_WRITER": pgstac_db.pgstac_secret.secret_value_from_json(
- "host"
- ).to_string(),
- "POSTGRES_DBNAME": pgstac_db.pgstac_secret.secret_value_from_json(
- "dbname"
- ).to_string(),
- "POSTGRES_USER": pgstac_db.pgstac_secret.secret_value_from_json(
- "username"
- ).to_string(),
- "POSTGRES_PASS": pgstac_db.pgstac_secret.secret_value_from_json(
- "password"
- ).to_string(),
- "POSTGRES_PORT": pgstac_db.pgstac_secret.secret_value_from_json(
- "port"
- ).to_string(),
- }
-
- eostac_settings = eoSTACSettings()
- env = eostac_settings.env or {}
- if "DB_MAX_CONN_SIZE" not in env:
- env["DB_MAX_CONN_SIZE"] = "1"
- if "DB_MIN_CONN_SIZE" not in env:
- env["DB_MIN_CONN_SIZE"] = "1"
- env.update(db_secrets)
- # If raster is deployed we had the TITILER_ENDPOINT env to add the Proxy extension
- if "raster" in eoapi_settings.functions:
- env["TITILER_ENDPOINT"] = eoraster.url.strip("/")
-
- eostac = PgStacApiLambda(
- self,
- id=f"{id}-stac-lambda",
- db=pgstac_db.db,
- db_secret=pgstac_db.pgstac_secret,
- api_env=env,
- lambda_function_options={
- "runtime": aws_lambda.Runtime.PYTHON_3_11,
- "code": aws_lambda.Code.from_docker_build(
- path=os.path.abspath(context_dir),
- file="infrastructure/aws/dockerfiles/Dockerfile.stac",
- build_args={
- "PYTHON_VERSION": "3.11",
- },
- platform="linux/amd64",
- ),
- "handler": "handler.handler",
- "memory_size": eostac_settings.memory,
- "timeout": Duration.seconds(eostac_settings.timeout),
- "log_retention": logs.RetentionDays.ONE_WEEK,
- },
- )
-
- if "browser" in eoapi_settings.functions:
- eobrowser_settings = eoStacBrowserSettings()
-
- stac_browser_bucket = s3.Bucket(
- self,
- "stac-browser-bucket",
- bucket_name=f"{id.lower()}-stac-browser",
- removal_policy=RemovalPolicy.DESTROY,
- auto_delete_objects=True,
- website_index_document="index.html",
- public_read_access=True,
- block_public_access=s3.BlockPublicAccess(
- block_public_acls=False,
- block_public_policy=False,
- ignore_public_acls=False,
- restrict_public_buckets=False,
- ),
- object_ownership=s3.ObjectOwnership.OBJECT_WRITER,
- )
-
- # need to build this manually, the attribute eostac.url is not resolved yet.
-
- StacBrowser(
- self,
- "stac-browser",
- github_repo_tag=eobrowser_settings.stac_browser_github_tag,
- stac_catalog_url=eobrowser_settings.stac_catalog_url,
- website_index_document="index.html",
- bucket_arn=stac_browser_bucket.bucket_arn,
- config_file_path=eobrowser_settings.config_file_path,
- )
-
- # eoapi.vector
- if "vector" in eoapi_settings.functions:
- db_secrets = {
- "POSTGRES_HOST": pgstac_db.pgstac_secret.secret_value_from_json(
- "host"
- ).to_string(),
- "POSTGRES_DBNAME": pgstac_db.pgstac_secret.secret_value_from_json(
- "dbname"
- ).to_string(),
- "POSTGRES_USER": pgstac_db.pgstac_secret.secret_value_from_json(
- "username"
- ).to_string(),
- "POSTGRES_PASS": pgstac_db.pgstac_secret.secret_value_from_json(
- "password"
- ).to_string(),
- "POSTGRES_PORT": pgstac_db.pgstac_secret.secret_value_from_json(
- "port"
- ).to_string(),
- }
-
- eovector_settings = eoVectorSettings()
- env = eovector_settings.env or {}
-
- if "DB_MAX_CONN_SIZE" not in env:
- env["DB_MAX_CONN_SIZE"] = "1"
- if "DB_MIN_CONN_SIZE" not in env:
- env["DB_MIN_CONN_SIZE"] = "1"
-
- env.update(db_secrets)
-
- TiPgApiLambda(
- self,
- f"{id}-vector-lambda",
- db=pgstac_db.db,
- db_secret=pgstac_db.pgstac_secret,
- api_env=env,
- lambda_function_options={
- "runtime": aws_lambda.Runtime.PYTHON_3_11,
- "code": aws_lambda.Code.from_docker_build(
- path=os.path.abspath(context_dir),
- file="infrastructure/aws/dockerfiles/Dockerfile.vector",
- build_args={
- "PYTHON_VERSION": "3.11",
- },
- platform="linux/amd64",
- ),
- "handler": "handler.handler",
- "memory_size": eovector_settings.memory,
- "timeout": Duration.seconds(eovector_settings.timeout),
- "log_retention": logs.RetentionDays.ONE_WEEK,
- },
- )
-
- if "ingestor" in eoapi_settings.functions:
-
- data_access_role = self._create_data_access_role()
-
- stac_ingestor = StacIngestor(
- self,
- "stac-ingestor",
- stac_url=eostac.url,
- stage=eoapi_settings.stage,
- data_access_role=data_access_role,
- stac_db_secret=pgstac_db.pgstac_secret,
- stac_db_security_group=pgstac_db.db.connections.security_groups[0],
- api_env={"REQUESTER_PAYS": "True"},
- )
-
- data_access_role = self._grant_assume_role_with_principal_pattern(
- data_access_role, stac_ingestor.handler_role.role_name
- )
-
- def _create_data_access_role(self) -> iam.Role:
-
- """
- Creates an IAM role with full S3 read access.
- """
-
- data_access_role = iam.Role(
- self,
- "data-access-role",
- assumed_by=iam.ServicePrincipal("lambda.amazonaws.com"),
- )
-
- data_access_role.add_managed_policy(
- iam.ManagedPolicy.from_aws_managed_policy_name("AmazonS3FullAccess")
- )
-
- return data_access_role
-
- def _grant_assume_role_with_principal_pattern(
- self,
- role_to_assume: iam.Role,
- principal_pattern: str,
- account_id: str = boto3.client("sts").get_caller_identity().get("Account"),
- ) -> iam.Role:
- """
- Grants assume role permissions to the role of the given
- account with the given name pattern. Default account
- is the current account.
- """
-
- role_to_assume.assume_role_policy.add_statements(
- iam.PolicyStatement(
- effect=iam.Effect.ALLOW,
- principals=[iam.AnyPrincipal()],
- actions=["sts:AssumeRole"],
- conditions={
- "StringLike": {
- "aws:PrincipalArn": [
- f"arn:aws:iam::{account_id}:role/{principal_pattern}"
- ]
- }
- },
- )
- )
-
- return role_to_assume
-
-
-app = App()
-
-
-eoapi_stack = eoAPIconstruct(
- app,
- f"{eoapi_settings.name}-{eoapi_settings.stage}",
- eoapi_settings.name,
- eoapi_settings.stage,
- env={
- "account": os.environ["CDK_DEFAULT_ACCOUNT"],
- "region": os.environ["CDK_DEFAULT_REGION"],
- },
-)
-
-# Tag infrastructure
-for key, value in {
- "Project": eoapi_settings.name,
- "Stack": eoapi_settings.stage,
- "Owner": eoapi_settings.owner,
- "Client": eoapi_settings.client,
-}.items():
- if value:
- Tags.of(eoapi_stack).add(key, value)
-
-
-app.synth()
diff --git a/infrastructure/aws/cdk/config.py b/infrastructure/aws/cdk/config.py
deleted file mode 100644
index 463a689..0000000
--- a/infrastructure/aws/cdk/config.py
+++ /dev/null
@@ -1,127 +0,0 @@
-"""eoAPI Configs."""
-
-from enum import Enum
-from typing import Dict, List, Optional
-
-from pydantic_settings import BaseSettings
-
-
-class functionName(str, Enum):
- """Function names."""
-
- stac = "stac"
- raster = "raster"
- vector = "vector"
- ingestor = "ingestor"
- browser = "browser" # not actually a function, but this keeps things clean.
-
-
-class eoAPISettings(BaseSettings):
- """Application settings"""
-
- name: str = "eoapi"
- stage: str = "production"
- owner: Optional[str] = None
- client: Optional[str] = None
- functions: List[functionName] = [functionName.stac, functionName.raster]
-
- model_config = {
- "env_prefix": "CDK_EOAPI_",
- "env_file": ".env",
- "extra": "ignore",
- "use_enum_values": True,
- }
-
-
-class eoDBSettings(BaseSettings):
- """Application settings"""
-
- dbname: str = "eoapi"
- user: str = "eouser"
-
- # Define PGSTAC VERSION
- pgstac_version: str
- instance_size: str = "SMALL"
- context: bool = True
- mosaic_index: bool = True
- allocated_storage: int = 20
- model_config = {
- "env_prefix": "CDK_EOAPI_DB_",
- "env_file": ".env",
- }
-
-
-class eoSTACSettings(BaseSettings):
- """Application settings"""
-
- env: Dict = {}
-
- timeout: int = 10
- memory: int = 256
- model_config = {
- "env_prefix": "CDK_EOAPI_STAC_",
- "env_file": ".env",
- }
-
-
-class eoRasterSettings(BaseSettings):
- """Application settings"""
-
- # Default options are optimized for CloudOptimized GeoTIFF
- # For more information on GDAL env see: https://gdal.org/user/configoptions.html
- # or https://developmentseed.org/titiler/advanced/performance_tuning/
- env: Dict = {
- "CPL_VSIL_CURL_ALLOWED_EXTENSIONS": ".tif,.TIF,.tiff",
- "GDAL_CACHEMAX": "200", # 200 mb
- "GDAL_DISABLE_READDIR_ON_OPEN": "EMPTY_DIR",
- "GDAL_INGESTED_BYTES_AT_OPEN": "32768",
- "GDAL_HTTP_MERGE_CONSECUTIVE_RANGES": "YES",
- "GDAL_HTTP_MULTIPLEX": "YES",
- "GDAL_HTTP_VERSION": "2",
- "PYTHONWARNINGS": "ignore",
- "VSI_CACHE": "TRUE",
- "VSI_CACHE_SIZE": "5000000", # 5 MB (per file-handle)
- "DB_MIN_CONN_SIZE": "1",
- "DB_MAX_CONN_SIZE": "1",
- }
-
- # S3 bucket names where TiTiler could do HEAD and GET Requests
- # specific private and public buckets MUST be added if you want to use s3:// urls
- # You can whitelist all bucket by setting `*`.
- # ref: https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-arn-format.html
- buckets: List = ["*"]
-
- timeout: int = 10
- memory: int = 3008
-
- model_config = {
- "env_prefix": "CDK_EOAPI_RASTER_",
- "env_file": ".env",
- }
-
-
-class eoVectorSettings(BaseSettings):
- """Application settings"""
-
- env: Dict = {}
-
- timeout: int = 10
- memory: int = 512
-
- model_config = {
- "env_prefix": "CDK_EOAPI_VECTOR_",
- "env_file": ".env",
- }
-
-
-class eoStacBrowserSettings(BaseSettings):
- """STAC browser settings"""
-
- stac_browser_github_tag: str = "v3.1.0"
- stac_catalog_url: str
- config_file_path: str = "browser_config.js"
-
- model_config = {
- "env_prefix": "CDK_EOAPI_BROWSER_",
- "env_file": ".env",
- }
diff --git a/infrastructure/aws/dockerfiles/Dockerfile.db b/infrastructure/aws/dockerfiles/Dockerfile.db
deleted file mode 100644
index 333fd3c..0000000
--- a/infrastructure/aws/dockerfiles/Dockerfile.db
+++ /dev/null
@@ -1,21 +0,0 @@
-ARG PYTHON_VERSION=3.10
-
-FROM public.ecr.aws/lambda/python:${PYTHON_VERSION}
-
-WORKDIR /tmp
-
-ARG PGSTAC_VERSION
-
-WORKDIR /tmp
-RUN python -m pip install pip -U
-
-RUN echo "Using PGSTAC Version ${PGSTAC_VERSION}"
-RUN python -m pip install httpx psycopg["binary,pool"] pypgstac==${PGSTAC_VERSION} -t /asset
-
-COPY infrastructure/aws/handlers/db_handler.py /asset/handler.py
-
-# https://stackoverflow.com/a/61746719
-# Turns out, asyncio is part of python
-RUN rm -rf /asset/asyncio*
-
-CMD ["echo", "hello world"]
diff --git a/infrastructure/aws/dockerfiles/Dockerfile.raster b/infrastructure/aws/dockerfiles/Dockerfile.raster
deleted file mode 100644
index 3259f54..0000000
--- a/infrastructure/aws/dockerfiles/Dockerfile.raster
+++ /dev/null
@@ -1,21 +0,0 @@
-ARG PYTHON_VERSION=3.10
-
-FROM public.ecr.aws/lambda/python:${PYTHON_VERSION}
-
-WORKDIR /tmp
-RUN python -m pip install pip -U
-
-COPY runtime/eoapi/raster /tmp/raster
-RUN python -m pip install "mangum>=0.14,<0.15" /tmp/raster["psycopg-binary"] -t /asset --no-binary pydantic
-RUN rm -rf /tmp/raster
-
-# Reduce package size and remove useless files
-RUN cd /asset && find . -type f -name '*.pyc' | while read f; do n=$(echo $f | sed 's/__pycache__\///' | sed 's/.cpython-[0-9]*//'); cp $f $n; done;
-RUN cd /asset && find . -type d -a -name '__pycache__' -print0 | xargs -0 rm -rf
-RUN cd /asset && find . -type f -a -name '*.py' -print0 | xargs -0 rm -f
-RUN find /asset -type d -a -name 'tests' -print0 | xargs -0 rm -rf
-RUN rm -rdf /asset/numpy/doc/ /asset/boto3* /asset/botocore* /asset/bin /asset/geos_license /asset/Misc
-
-COPY infrastructure/aws/handlers/raster_handler.py /asset/handler.py
-
-CMD ["echo", "hello world"]
diff --git a/infrastructure/aws/dockerfiles/Dockerfile.stac b/infrastructure/aws/dockerfiles/Dockerfile.stac
deleted file mode 100644
index a9675e7..0000000
--- a/infrastructure/aws/dockerfiles/Dockerfile.stac
+++ /dev/null
@@ -1,20 +0,0 @@
-ARG PYTHON_VERSION=3.10
-
-FROM public.ecr.aws/lambda/python:${PYTHON_VERSION}
-
-WORKDIR /tmp
-RUN python -m pip install pip -U
-
-COPY runtime/eoapi/stac /tmp/stac
-RUN python -m pip install "mangum>=0.14,<0.15" /tmp/stac -t /asset --no-binary pydantic
-RUN rm -rf /tmp/stac
-
-# Reduce package size and remove useless files
-RUN cd /asset && find . -type f -name '*.pyc' | while read f; do n=$(echo $f | sed 's/__pycache__\///' | sed 's/.cpython-[0-9]*//'); cp $f $n; done;
-RUN cd /asset && find . -type d -a -name '__pycache__' -print0 | xargs -0 rm -rf
-RUN cd /asset && find . -type f -a -name '*.py' -print0 | xargs -0 rm -f
-RUN find /asset -type d -a -name 'tests' -print0 | xargs -0 rm -rf
-
-COPY infrastructure/aws/handlers/stac_handler.py /asset/handler.py
-
-CMD ["echo", "hello world"]
diff --git a/infrastructure/aws/dockerfiles/Dockerfile.vector b/infrastructure/aws/dockerfiles/Dockerfile.vector
deleted file mode 100644
index eb211f9..0000000
--- a/infrastructure/aws/dockerfiles/Dockerfile.vector
+++ /dev/null
@@ -1,22 +0,0 @@
-ARG PYTHON_VERSION=3.10
-
-FROM public.ecr.aws/lambda/python:${PYTHON_VERSION}
-
-RUN yum install -y git
-
-WORKDIR /tmp
-RUN python -m pip install pip -U
-
-COPY runtime/eoapi/vector /tmp/vector
-RUN python -m pip install "mangum>=0.14,<0.15" /tmp/vector -t /asset --no-binary pydantic
-RUN rm -rf /tmp/vector
-
-# Reduce package size and remove useless files
-RUN cd /asset && find . -type f -name '*.pyc' | while read f; do n=$(echo $f | sed 's/__pycache__\///' | sed 's/.cpython-[0-9]*//'); cp $f $n; done;
-RUN cd /asset && find . -type d -a -name '__pycache__' -print0 | xargs -0 rm -rf
-RUN cd /asset && find . -type f -a -name '*.py' -print0 | xargs -0 rm -f
-RUN find /asset -type d -a -name 'tests' -print0 | xargs -0 rm -rf
-
-COPY infrastructure/aws/handlers/vector_handler.py /asset/handler.py
-
-CMD ["echo", "hello world"]
diff --git a/infrastructure/aws/handlers/db_handler.py b/infrastructure/aws/handlers/db_handler.py
deleted file mode 100644
index 4ec0847..0000000
--- a/infrastructure/aws/handlers/db_handler.py
+++ /dev/null
@@ -1,275 +0,0 @@
-"""Bootstrap Postgres db."""
-
-import json
-import logging
-
-import boto3
-import httpx
-import psycopg
-from psycopg import sql
-from psycopg.conninfo import make_conninfo
-from pypgstac.db import PgstacDB
-from pypgstac.migrate import Migrate
-
-logger = logging.getLogger("eoapi-bootstrap")
-
-
-def send(
- event,
- context,
- responseStatus,
- responseData,
- physicalResourceId=None,
- noEcho=False,
-):
- """
- Copyright 2016 Amazon Web Services, Inc. or its affiliates. All Rights Reserved.
- This file is licensed to you under the AWS Customer Agreement (the "License").
- You may not use this file except in compliance with the License.
- A copy of the License is located at http://aws.amazon.com/agreement/ .
- This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied.
- See the License for the specific language governing permissions and limitations under the License.
-
- Send response from AWS Lambda.
-
- Note: The cfnresponse module is available only when you use the ZipFile property to write your source code.
- It isn't available for source code that's stored in Amazon S3 buckets.
- For code in buckets, you must write your own functions to send responses.
- """
- responseUrl = event["ResponseURL"]
-
- print(responseUrl)
-
- responseBody = {}
- responseBody["Status"] = responseStatus
- responseBody["Reason"] = (
- "See the details in CloudWatch Log Stream: " + context.log_stream_name
- )
- responseBody["PhysicalResourceId"] = physicalResourceId or context.log_stream_name
- responseBody["StackId"] = event["StackId"]
- responseBody["RequestId"] = event["RequestId"]
- responseBody["LogicalResourceId"] = event["LogicalResourceId"]
- responseBody["NoEcho"] = noEcho
- responseBody["Data"] = responseData
-
- json_responseBody = json.dumps(responseBody)
-
- print("Response body:\n" + json_responseBody)
-
- headers = {"content-type": "", "content-length": str(len(json_responseBody))}
-
- try:
- response = httpx.put(responseUrl, data=json_responseBody, headers=headers)
- print(f"Status code: {response.status_code}")
- logger.debug(f"OK - Status code: {response.status_code}")
- except Exception as e:
- print("send(..) failed executing httpx.put(..): " + str(e))
- logger.debug(f"NOK - failed executing PUT requests: {e}")
-
-
-def get_secret(secret_name):
- """Get Secrets from secret manager."""
- print(f"Fetching {secret_name}")
- client = boto3.client(service_name="secretsmanager")
- response = client.get_secret_value(SecretId=secret_name)
- return json.loads(response["SecretString"])
-
-
-def create_db(cursor, db_name: str) -> None:
- """Create DB."""
- cursor.execute(
- sql.SQL("SELECT 1 FROM pg_catalog.pg_database " "WHERE datname = %s"), [db_name]
- )
- if cursor.fetchone():
- print(f"database {db_name} exists, not creating DB")
- else:
- print(f"database {db_name} not found, creating...")
- cursor.execute(
- sql.SQL("CREATE DATABASE {db_name}").format(db_name=sql.Identifier(db_name))
- )
-
-
-def create_user(cursor, username: str, password: str) -> None:
- """Create User."""
- cursor.execute(
- sql.SQL(
- "DO $$ "
- "BEGIN "
- " IF NOT EXISTS ( "
- " SELECT 1 FROM pg_roles "
- " WHERE rolname = {user}) "
- " THEN "
- " CREATE USER {username} "
- " WITH PASSWORD {password}; "
- " ELSE "
- " ALTER USER {username} "
- " WITH PASSWORD {password}; "
- " END IF; "
- "END "
- "$$; "
- ).format(username=sql.Identifier(username), password=password, user=username)
- )
-
-
-def create_permissions(cursor, db_name: str, username: str) -> None:
- """Add permissions."""
- cursor.execute(
- sql.SQL(
- "GRANT CONNECT ON DATABASE {db_name} TO {username};"
- "GRANT CREATE ON DATABASE {db_name} TO {username};" # Allow schema creation
- "GRANT USAGE ON SCHEMA public TO {username};"
- "ALTER DEFAULT PRIVILEGES IN SCHEMA public "
- "GRANT ALL PRIVILEGES ON TABLES TO {username};"
- "ALTER DEFAULT PRIVILEGES IN SCHEMA public "
- "GRANT ALL PRIVILEGES ON SEQUENCES TO {username};"
- "GRANT pgstac_read TO {username};"
- "GRANT pgstac_ingest TO {username};"
- "GRANT pgstac_admin TO {username};"
- ).format(
- db_name=sql.Identifier(db_name),
- username=sql.Identifier(username),
- )
- )
-
-
-def register_extensions(cursor) -> None:
- """Add PostGIS extension."""
- cursor.execute(sql.SQL("CREATE EXTENSION IF NOT EXISTS postgis;"))
-
-
-###############################################################################
-# PgSTAC Customization
-###############################################################################
-def customization(cursor, params) -> None:
- """
- CUSTOMIZED YOUR PGSTAC DATABASE
-
- ref: https://github.com/stac-utils/pgstac/blob/main/docs/src/pgstac.md
-
- """
- if params.get("context", False):
- # Add CONTEXT=ON
- pgstac_settings = """
- INSERT INTO pgstac_settings (name, value)
- VALUES ('context', 'on')
- ON CONFLICT ON CONSTRAINT pgstac_settings_pkey DO UPDATE SET value = excluded.value;"""
- cursor.execute(sql.SQL(pgstac_settings))
-
- if params.get("mosaic_index", False):
- # Create index of searches with `mosaic`` type
- cursor.execute(
- sql.SQL(
- "CREATE INDEX IF NOT EXISTS searches_mosaic ON searches ((true)) WHERE metadata->>'type'='mosaic';"
- )
- )
-
-
-def handler(event, context):
- """Lambda Handler."""
- print(f"Handling {event}")
-
- if event["RequestType"] not in ["Create", "Update"]:
- return send(event, context, "SUCCESS", {"msg": "No action to be taken"})
-
- try:
- params = event["ResourceProperties"]
- connection_params = get_secret(params["conn_secret_arn"])
- user_params = get_secret(params["new_user_secret_arn"])
-
- print("Connecting to admin DB...")
- admin_db_conninfo = make_conninfo(
- dbname=connection_params.get("dbname", "postgres"),
- user=connection_params["username"],
- password=connection_params["password"],
- host=connection_params["host"],
- port=connection_params["port"],
- )
- with psycopg.connect(admin_db_conninfo, autocommit=True) as conn:
- with conn.cursor() as cur:
- print("Creating database...")
- create_db(
- cursor=cur,
- db_name=user_params["dbname"],
- )
-
- print("Creating user...")
- create_user(
- cursor=cur,
- username=user_params["username"],
- password=user_params["password"],
- )
-
- # Install extensions on the user DB with
- # superuser permissions, since they will
- # otherwise fail to install when run as
- # the non-superuser within the pgstac
- # migrations.
- print("Connecting to STAC DB...")
- stac_db_conninfo = make_conninfo(
- dbname=user_params["dbname"],
- user=connection_params["username"],
- password=connection_params["password"],
- host=connection_params["host"],
- port=connection_params["port"],
- )
- with psycopg.connect(stac_db_conninfo, autocommit=True) as conn:
- with conn.cursor() as cur:
- print("Registering PostGIS ...")
- register_extensions(cursor=cur)
-
- stac_db_admin_dsn = (
- "postgresql://{user}:{password}@{host}:{port}/{dbname}".format(
- dbname=user_params.get("dbname", "postgres"),
- user=connection_params["username"],
- password=connection_params["password"],
- host=connection_params["host"],
- port=connection_params["port"],
- )
- )
-
- with PgstacDB(dsn=stac_db_admin_dsn, debug=True) as pgdb:
- print(f"Current {pgdb.version}")
-
- # As admin, run migrations
- print("Running migrations...")
- Migrate(pgdb).run_migration(params["pgstac_version"])
-
- # Assign appropriate permissions to user (requires pgSTAC migrations to have run)
- with psycopg.connect(admin_db_conninfo, autocommit=True) as conn:
- with conn.cursor() as cur:
- print("Setting permissions...")
- create_permissions(
- cursor=cur,
- db_name=user_params["dbname"],
- username=user_params["username"],
- )
-
- print("Customize PgSTAC database...")
- with psycopg.connect(
- stac_db_admin_dsn,
- autocommit=True,
- options="-c search_path=pgstac,public -c application_name=pgstac",
- ) as conn:
- with conn.cursor() as cur:
- customization(cursor=cur, params=params)
-
- # Make sure the user can access the database
- stac_db_user_dsn = (
- "postgresql://{user}:{password}@{host}:{port}/{dbname}".format(
- dbname=user_params.get("dbname", "postgres"),
- user=user_params["username"],
- password=user_params["password"],
- host=connection_params["host"],
- port=connection_params["port"],
- )
- )
- with PgstacDB(dsn=stac_db_user_dsn, debug=True) as pgdb:
- print(f"User can access pgstac: {pgdb.version}")
-
- except Exception as e:
- print(f"Unable to bootstrap database with exception={e}")
- send(event, context, "FAILED", {"message": str(e)})
- raise e
-
- print("Complete.")
- return send(event, context, "SUCCESS", {})
diff --git a/infrastructure/aws/handlers/raster_handler.py b/infrastructure/aws/handlers/raster_handler.py
deleted file mode 100644
index 36da4f7..0000000
--- a/infrastructure/aws/handlers/raster_handler.py
+++ /dev/null
@@ -1,25 +0,0 @@
-"""AWS Lambda handler."""
-
-import asyncio
-import logging
-import os
-
-from eoapi.raster.app import app
-from mangum import Mangum
-from titiler.pgstac.db import connect_to_db
-
-logging.getLogger("mangum.lifespan").setLevel(logging.ERROR)
-logging.getLogger("mangum.http").setLevel(logging.ERROR)
-
-
-@app.on_event("startup")
-async def startup_event() -> None:
- """Connect to database on startup."""
- await connect_to_db(app)
-
-
-handler = Mangum(app, lifespan="off")
-
-if "AWS_EXECUTION_ENV" in os.environ:
- loop = asyncio.get_event_loop()
- loop.run_until_complete(app.router.startup())
diff --git a/infrastructure/aws/handlers/stac_handler.py b/infrastructure/aws/handlers/stac_handler.py
deleted file mode 100644
index 1bc4a66..0000000
--- a/infrastructure/aws/handlers/stac_handler.py
+++ /dev/null
@@ -1,25 +0,0 @@
-"""AWS Lambda handler."""
-
-import asyncio
-import logging
-import os
-
-from eoapi.stac.app import app
-from mangum import Mangum
-from stac_fastapi.pgstac.db import connect_to_db
-
-logging.getLogger("mangum.lifespan").setLevel(logging.ERROR)
-logging.getLogger("mangum.http").setLevel(logging.ERROR)
-
-
-@app.on_event("startup")
-async def startup_event() -> None:
- """Connect to database on startup."""
- await connect_to_db(app)
-
-
-handler = Mangum(app, lifespan="off")
-
-if "AWS_EXECUTION_ENV" in os.environ:
- loop = asyncio.get_event_loop()
- loop.run_until_complete(app.router.startup())
diff --git a/infrastructure/aws/handlers/vector_handler.py b/infrastructure/aws/handlers/vector_handler.py
deleted file mode 100644
index c7ea2ea..0000000
--- a/infrastructure/aws/handlers/vector_handler.py
+++ /dev/null
@@ -1,54 +0,0 @@
-"""AWS Lambda handler."""
-
-import asyncio
-import logging
-import os
-
-from eoapi.vector.app import app
-from mangum import Mangum
-from tipg.collections import register_collection_catalog
-from tipg.database import connect_to_db
-from tipg.settings import PostgresSettings
-
-logging.getLogger("mangum.lifespan").setLevel(logging.ERROR)
-logging.getLogger("mangum.http").setLevel(logging.ERROR)
-
-postgres_settings = PostgresSettings()
-
-try:
- from importlib.resources import files as resources_files # type: ignore
-except ImportError:
- # Try backported to PY<39 `importlib_resources`.
- from importlib_resources import files as resources_files # type: ignore
-
-
-CUSTOM_SQL_DIRECTORY = resources_files("eoapi.vector") / "sql"
-sql_files = list(CUSTOM_SQL_DIRECTORY.glob("*.sql")) # type: ignore
-
-
-@app.on_event("startup")
-async def startup_event() -> None:
- """Connect to database on startup."""
- await connect_to_db(
- app,
- settings=postgres_settings,
- # We enable both pgstac and public schemas (pgstac will be used by custom functions)
- schemas=["pgstac", "public"],
- user_sql_files=sql_files,
- )
- await register_collection_catalog(
- app,
- # For the Tables' Catalog we only use the `public` schema
- schemas=["public"],
- # We exclude public functions
- exclude_function_schemas=["public"],
- # We allow non-spatial tables
- spatial=False,
- )
-
-
-handler = Mangum(app, lifespan="off")
-
-if "AWS_EXECUTION_ENV" in os.environ:
- loop = asyncio.get_event_loop()
- loop.run_until_complete(app.router.startup())
diff --git a/infrastructure/aws/package-lock.json b/infrastructure/aws/package-lock.json
deleted file mode 100644
index f1aa3a8..0000000
--- a/infrastructure/aws/package-lock.json
+++ /dev/null
@@ -1,81 +0,0 @@
-{
- "name": "cdk-deploy",
- "version": "0.1.0",
- "lockfileVersion": 2,
- "requires": true,
- "packages": {
- "": {
- "name": "cdk-deploy",
- "version": "0.1.0",
- "license": "MIT",
- "dependencies": {
- "cdk": "2.94.0"
- }
- },
- "node_modules/aws-cdk": {
- "version": "2.94.0",
- "resolved": "https://registry.npmjs.org/aws-cdk/-/aws-cdk-2.94.0.tgz",
- "integrity": "sha512-9bJkzxFDYZDwPDfZi/DSUODn4HFRzuXWPhpFgIIgRykfT18P+iAIJ1AEhaaCmlqrrog5yQgN+2iYd9BwDsiBeg==",
- "bin": {
- "cdk": "bin/cdk"
- },
- "engines": {
- "node": ">= 14.15.0"
- },
- "optionalDependencies": {
- "fsevents": "2.3.2"
- }
- },
- "node_modules/cdk": {
- "version": "2.94.0",
- "resolved": "https://registry.npmjs.org/cdk/-/cdk-2.94.0.tgz",
- "integrity": "sha512-dMgSTaMtfpPxY2biMSHlNrwrJcq0iJkihvkVuSSQyhlHyLmH0s9fSBzO8zrGFAoEp/cDofg6iDfGzmwrHQ55LA==",
- "dependencies": {
- "aws-cdk": "2.94.0"
- },
- "bin": {
- "cdk": "bin/cdk"
- },
- "engines": {
- "node": ">= 14.15.0"
- }
- },
- "node_modules/fsevents": {
- "version": "2.3.2",
- "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
- "integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
- "hasInstallScript": true,
- "optional": true,
- "os": [
- "darwin"
- ],
- "engines": {
- "node": "^8.16.0 || ^10.6.0 || >=11.0.0"
- }
- }
- },
- "dependencies": {
- "aws-cdk": {
- "version": "2.94.0",
- "resolved": "https://registry.npmjs.org/aws-cdk/-/aws-cdk-2.94.0.tgz",
- "integrity": "sha512-9bJkzxFDYZDwPDfZi/DSUODn4HFRzuXWPhpFgIIgRykfT18P+iAIJ1AEhaaCmlqrrog5yQgN+2iYd9BwDsiBeg==",
- "requires": {
- "fsevents": "2.3.2"
- }
- },
- "cdk": {
- "version": "2.94.0",
- "resolved": "https://registry.npmjs.org/cdk/-/cdk-2.94.0.tgz",
- "integrity": "sha512-dMgSTaMtfpPxY2biMSHlNrwrJcq0iJkihvkVuSSQyhlHyLmH0s9fSBzO8zrGFAoEp/cDofg6iDfGzmwrHQ55LA==",
- "requires": {
- "aws-cdk": "2.94.0"
- }
- },
- "fsevents": {
- "version": "2.3.2",
- "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
- "integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
- "optional": true
- }
- }
-}
diff --git a/infrastructure/aws/package.json b/infrastructure/aws/package.json
deleted file mode 100644
index f551ad4..0000000
--- a/infrastructure/aws/package.json
+++ /dev/null
@@ -1,13 +0,0 @@
-{
- "name": "cdk-deploy",
- "version": "0.1.0",
- "description": "Dependencies for CDK deployment",
- "license": "MIT",
- "private": true,
- "dependencies": {
- "cdk": "2.94.0"
- },
- "scripts": {
- "cdk": "cdk"
- }
-}
diff --git a/infrastructure/aws/requirements-cdk.txt b/infrastructure/aws/requirements-cdk.txt
deleted file mode 100644
index 867a9f7..0000000
--- a/infrastructure/aws/requirements-cdk.txt
+++ /dev/null
@@ -1,10 +0,0 @@
-# aws cdk
-aws-cdk-lib==2.99.1
-aws_cdk-aws_apigatewayv2_alpha==2.99.1a0
-aws_cdk-aws_apigatewayv2_integrations_alpha==2.99.1a0
-constructs>=10.0.0
-boto3==1.28.71
-# pydantic settings
-pydantic~=2.0
-pydantic-settings~=2.0
-eoapi-cdk==6.1.0
\ No newline at end of file
diff --git a/infrastructure/aws/tests/conftest.py b/infrastructure/aws/tests/conftest.py
deleted file mode 100644
index e2190ed..0000000
--- a/infrastructure/aws/tests/conftest.py
+++ /dev/null
@@ -1,19 +0,0 @@
-"""``pytest`` configuration."""
-
-
-import pytest
-import pytest_pgsql
-
-test_db = pytest_pgsql.TransactedPostgreSQLTestDB.create_fixture(
- "test_db", scope="session", use_restore_state=False
-)
-
-
-@pytest.fixture(scope="session")
-def database_url(test_db):
- """
- Session scoped fixture to launch a postgresql database in a separate process. We use psycopg2 to ingest test data
- because pytest-asyncio event loop is a function scoped fixture and cannot be called within the current scope. Yields
- a database url which we pass to our application through a monkeypatched environment variable.
- """
- return test_db.connection.engine.url
diff --git a/infrastructure/aws/tests/test_bootstrap.py b/infrastructure/aws/tests/test_bootstrap.py
deleted file mode 100644
index bc72460..0000000
--- a/infrastructure/aws/tests/test_bootstrap.py
+++ /dev/null
@@ -1,74 +0,0 @@
-"""Test bootstrap handler."""
-
-import json
-import logging
-from unittest.mock import patch
-
-from handlers import db_handler
-
-
-@patch("handlers.db_handler.httpx")
-@patch("handlers.db_handler.boto3")
-def test_bootstrap(boto3, httpx, database_url, caplog):
- """Test Bootstrap."""
- boto3.client.return_value.get_secret_value.side_effect = [
- # connection_params
- {
- "SecretString": json.dumps(
- {
- "dbname": "postgres",
- "username": database_url.username,
- "password": database_url.password,
- "host": database_url.host,
- "port": database_url.port,
- }
- )
- },
- # user_params
- {
- "SecretString": json.dumps(
- {
- "dbname": "eoapi",
- "username": "eouser",
- "password": "mypassword",
- "host": database_url.host,
- "port": database_url.port,
- }
- )
- },
- ]
-
- class put:
- """Fake httpx.put response."""
-
- status_code: int = 200
- reason: str = "All Good"
-
- httpx.put.return_value = put()
-
- event = {
- "StackId": "eoapi-test",
- "RequestId": "request-00001",
- "LogicalResourceId": "eoapi-test",
- "RequestType": "Create",
- "ResourceProperties": {
- "pgstac_version": "0.6.11",
- "context": True,
- "mosaic_index": True,
- "conn_secret_arn": "secret1",
- "new_user_secret_arn": "secret2",
- },
- "ResponseURL": "http://0.0.0.0",
- }
-
- class ctx:
- """Fake context object."""
-
- log_stream_name: str = "something"
-
- context = ctx()
-
- with caplog.at_level(logging.DEBUG, logger="eoapi-bootstrap"):
- db_handler.handler(event, context)
- for record in caplog.records:
- assert record.message == "OK - Status code: 200"
diff --git a/ruff.toml b/ruff.toml
deleted file mode 100644
index d92bf79..0000000
--- a/ruff.toml
+++ /dev/null
@@ -1,13 +0,0 @@
-select = [
- "D1", # pydocstyle errors
- "E", # pycodestyle errors
- "W", # pycodestyle warnings
- "F", # flake8
- "C", # flake8-comprehensions
- "B", # flake8-bugbear
-]
-ignore = [
- "E501", # line too long, handled by black
- "B008", # do not perform function calls in argument defaults
- "B905", # ignore zip() without an explicit strict= parameter, only support with python >3.10
-]
diff --git a/runtime/eoapi/raster/README.md b/runtime/eoapi/raster/README.md
deleted file mode 100644
index aeefc7e..0000000
--- a/runtime/eoapi/raster/README.md
+++ /dev/null
@@ -1,3 +0,0 @@
-## eoapi.raster
-
-
diff --git a/runtime/eoapi/raster/eoapi/raster/__init__.py b/runtime/eoapi/raster/eoapi/raster/__init__.py
deleted file mode 100644
index 9b25578..0000000
--- a/runtime/eoapi/raster/eoapi/raster/__init__.py
+++ /dev/null
@@ -1,3 +0,0 @@
-"""eoapi.raster."""
-
-__version__ = "0.1.0"
diff --git a/runtime/eoapi/raster/eoapi/raster/app.py b/runtime/eoapi/raster/eoapi/raster/app.py
deleted file mode 100644
index 4012d16..0000000
--- a/runtime/eoapi/raster/eoapi/raster/app.py
+++ /dev/null
@@ -1,370 +0,0 @@
-"""TiTiler+PgSTAC FastAPI application."""
-
-import logging
-from contextlib import asynccontextmanager
-from typing import Dict
-
-import jinja2
-import pystac
-from eoapi.raster import __version__ as eoapi_raster_version
-from eoapi.raster.config import ApiSettings
-from fastapi import Depends, FastAPI, Query
-from psycopg import OperationalError
-from psycopg.rows import dict_row
-from psycopg_pool import PoolTimeout
-from starlette.middleware.cors import CORSMiddleware
-from starlette.requests import Request
-from starlette.responses import HTMLResponse
-from starlette.templating import Jinja2Templates
-from starlette_cramjam.middleware import CompressionMiddleware
-from titiler.core.errors import DEFAULT_STATUS_CODES, add_exception_handlers
-from titiler.core.factory import (
- AlgorithmFactory,
- MultiBaseTilerFactory,
- TilerFactory,
- TMSFactory,
-)
-from titiler.core.middleware import CacheControlMiddleware
-from titiler.extensions import cogViewerExtension
-from titiler.mosaic.errors import MOSAIC_STATUS_CODES
-from titiler.pgstac.db import close_db_connection, connect_to_db
-from titiler.pgstac.dependencies import CollectionIdParams, ItemIdParams, SearchIdParams
-from titiler.pgstac.extensions import searchInfoExtension
-from titiler.pgstac.factory import (
- MosaicTilerFactory,
- add_search_list_route,
- add_search_register_route,
-)
-from titiler.pgstac.reader import PgSTACReader
-
-logging.getLogger("botocore.credentials").disabled = True
-logging.getLogger("botocore.utils").disabled = True
-logging.getLogger("rio-tiler").setLevel(logging.ERROR)
-
-settings = ApiSettings()
-
-jinja2_env = jinja2.Environment(
- loader=jinja2.ChoiceLoader(
- [
- jinja2.PackageLoader(__package__, "templates"),
- ]
- )
-)
-templates = Jinja2Templates(env=jinja2_env)
-
-
-@asynccontextmanager
-async def lifespan(app: FastAPI):
- """FastAPI Lifespan."""
- # Create Connection Pool
- await connect_to_db(app)
- yield
- # Close the Connection Pool
- await close_db_connection(app)
-
-
-app = FastAPI(
- title=settings.name,
- version=eoapi_raster_version,
- root_path=settings.root_path,
- lifespan=lifespan,
-)
-add_exception_handlers(app, DEFAULT_STATUS_CODES)
-add_exception_handlers(app, MOSAIC_STATUS_CODES)
-
-if settings.cors_origins:
- app.add_middleware(
- CORSMiddleware,
- allow_origins=settings.cors_origins,
- allow_credentials=True,
- allow_methods=["GET", "POST", "OPTIONS"],
- allow_headers=["*"],
- )
-
-app.add_middleware(
- CacheControlMiddleware,
- cachecontrol=settings.cachecontrol,
- exclude_path={r"/healthz"},
-)
-app.add_middleware(
- CompressionMiddleware,
- exclude_mediatype={
- "image/jpeg",
- "image/jpg",
- "image/png",
- "image/jp2",
- "image/webp",
- },
-)
-
-###############################################################################
-# `Secret` endpoint for mosaic builder. Do not need to be public (in the OpenAPI docs)
-@app.get("/collections", include_in_schema=False)
-async def list_collection(request: Request):
- """list collections."""
- with request.app.state.dbpool.connection() as conn:
- with conn.cursor(row_factory=dict_row) as cursor:
- cursor.execute("SELECT * FROM pgstac.all_collections();")
- r = cursor.fetchone()
- return r.get("all_collections", [])
-
-
-###############################################################################
-# STAC Search Endpoints
-searches = MosaicTilerFactory(
- path_dependency=SearchIdParams,
- router_prefix="/searches/{search_id}",
- add_statistics=True,
- add_viewer=True,
- add_part=True,
- extensions=[
- searchInfoExtension(),
- ],
-)
-app.include_router(
- searches.router, tags=["STAC Search"], prefix="/searches/{search_id}"
-)
-
-add_search_register_route(
- app,
- prefix="/searches",
- tile_dependencies=[
- searches.layer_dependency,
- searches.dataset_dependency,
- searches.pixel_selection_dependency,
- searches.tile_dependency,
- searches.process_dependency,
- searches.rescale_dependency,
- searches.colormap_dependency,
- searches.render_dependency,
- searches.pgstac_dependency,
- searches.reader_dependency,
- searches.backend_dependency,
- ],
- tags=["STAC Search"],
-)
-add_search_list_route(app, prefix="/searches", tags=["STAC Search"])
-
-
-@app.get("/searches/builder", response_class=HTMLResponse, tags=["STAC Search"])
-async def virtual_mosaic_builder(request: Request):
- """Mosaic Builder Viewer."""
- base_url = str(request.base_url)
- return templates.TemplateResponse(
- name="mosaic-builder.html",
- context={
- "request": request,
- "register_endpoint": str(
- app.url_path_for("register_search").make_absolute_url(base_url=base_url)
- ),
- "collections_endpoint": str(
- app.url_path_for("list_collection").make_absolute_url(base_url=base_url)
- ),
- },
- media_type="text/html",
- )
-
-
-###############################################################################
-# STAC COLLECTION Endpoints
-collection = MosaicTilerFactory(
- path_dependency=CollectionIdParams,
- router_prefix="/collections/{collection_id}",
- add_statistics=True,
- add_viewer=True,
- add_part=True,
-)
-app.include_router(
- collection.router, tags=["STAC Collection"], prefix="/collections/{collection_id}"
-)
-
-
-###############################################################################
-# STAC Item Endpoints
-stac = MultiBaseTilerFactory(
- reader=PgSTACReader,
- path_dependency=ItemIdParams,
- router_prefix="/collections/{collection_id}/items/{item_id}",
- add_viewer=True,
-)
-app.include_router(
- stac.router,
- tags=["STAC Item"],
- prefix="/collections/{collection_id}/items/{item_id}",
-)
-
-
-@stac.router.get("/viewer", response_class=HTMLResponse)
-def viewer(request: Request, item: pystac.Item = Depends(stac.path_dependency)):
- """STAC Viewer
-
- Simplified version of https://github.com/developmentseed/titiler/blob/main/src/titiler/extensions/titiler/extensions/templates/stac_viewer.html
- """
- return templates.TemplateResponse(
- name="stac-viewer.html",
- context={
- "request": request,
- "endpoint": request.url.path.replace("/viewer", ""),
- },
- media_type="text/html",
- )
-
-
-app.include_router(
- stac.router,
- tags=["STAC Item"],
- prefix="/collections/{collection_id}/items/{item_id}",
-)
-
-
-###############################################################################
-# COG Endpoints
-cog = TilerFactory(
- router_prefix="/cog",
- extensions=[
- cogViewerExtension(),
- ],
-)
-
-app.include_router(cog.router, prefix="/cog", tags=["Cloud Optimized GeoTIFF"])
-
-###############################################################################
-# Tiling Schemes Endpoints
-tms = TMSFactory()
-app.include_router(tms.router, tags=["Tiling Schemes"])
-
-###############################################################################
-# Algorithms Endpoints
-algorithms = AlgorithmFactory()
-app.include_router(algorithms.router, tags=["Algorithms"])
-
-
-###############################################################################
-# Health Check Endpoint
-@app.get("/healthz", description="Health Check", tags=["Health Check"])
-def ping(
- timeout: int = Query(1, description="Timeout getting SQL connection from the pool.")
-) -> Dict:
- """Health check."""
- try:
- with app.state.dbpool.connection(timeout) as conn:
- with conn.cursor() as cursor:
- cursor.execute("SELECT version from pgstac.migrations;")
- version = cursor.fetchone()
- return {"database_online": True, "pgstac_version": version}
- except (OperationalError, PoolTimeout):
- return {"database_online": False}
-
-
-###############################################################################
-# Landing page Endpoint
-@app.get(
- "/",
- response_class=HTMLResponse,
-)
-def landing(request: Request):
- """Get landing page."""
- data = {
- "title": settings.name or "eoAPI-raster",
- "links": [
- {
- "title": "Landing page",
- "href": str(request.url_for("landing")),
- "type": "text/html",
- "rel": "self",
- },
- {
- "title": "the API definition (JSON)",
- "href": str(request.url_for("openapi")),
- "type": "application/vnd.oai.openapi+json;version=3.0",
- "rel": "service-desc",
- },
- {
- "title": "the API documentation",
- "href": str(request.url_for("swagger_ui_html")),
- "type": "text/html",
- "rel": "service-doc",
- },
- {
- "title": "PgSTAC Virtual Mosaic list (JSON)",
- "href": str(app.url_path_for("list_searches")),
- "type": "application/json",
- "rel": "data",
- },
- {
- "title": "PgSTAC Virtual Mosaic builder",
- "href": str(app.url_path_for("virtual_mosaic_builder")),
- "type": "text/html",
- "rel": "data",
- },
- {
- "title": "PgSTAC Virtual Mosaic viewer (template URL)",
- "href": str(app.url_path_for("map_viewer", search_id="{search_id}")),
- "type": "text/html",
- "rel": "data",
- },
- {
- "title": "PgSTAC Collection viewer (template URL)",
- "href": str(
- app.url_path_for("map_viewer", collection_id="{collection_id}")
- ),
- "type": "text/html",
- "rel": "data",
- },
- {
- "title": "PgSTAC Item viewer (template URL)",
- "href": str(
- app.url_path_for(
- "map_viewer",
- collection_id="{collection_id}",
- item_id="{item_id}",
- )
- ),
- "type": "text/html",
- "rel": "data",
- },
- {
- "title": "TiTiler-PgSTAC Documentation (external link)",
- "href": "https://stac-utils.github.io/titiler-pgstac/",
- "type": "text/html",
- "rel": "doc",
- },
- {
- "title": "TiTiler-PgSTAC source code (external link)",
- "href": "https://github.com/stac-utils/titiler-pgstac",
- "type": "text/html",
- "rel": "doc",
- },
- ],
- }
-
- urlpath = request.url.path
- crumbs = []
- baseurl = str(request.base_url).rstrip("/")
-
- crumbpath = str(baseurl)
- for crumb in urlpath.split("/"):
- crumbpath = crumbpath.rstrip("/")
- part = crumb
- if part is None or part == "":
- part = "Home"
- crumbpath += f"/{crumb}"
- crumbs.append({"url": crumbpath.rstrip("/"), "part": part.capitalize()})
-
- return templates.TemplateResponse(
- "landing.html",
- {
- "request": request,
- "response": data,
- "template": {
- "api_root": baseurl,
- "params": request.query_params,
- "title": "",
- },
- "crumbs": crumbs,
- "url": str(request.url),
- "baseurl": baseurl,
- "urlpath": str(request.url.path),
- "urlparams": str(request.url.query),
- },
- )
diff --git a/runtime/eoapi/raster/eoapi/raster/config.py b/runtime/eoapi/raster/eoapi/raster/config.py
deleted file mode 100644
index f6f898b..0000000
--- a/runtime/eoapi/raster/eoapi/raster/config.py
+++ /dev/null
@@ -1,21 +0,0 @@
-"""API settings."""
-
-from pydantic import field_validator
-from pydantic_settings import BaseSettings
-
-
-class ApiSettings(BaseSettings):
- """API settings"""
-
- name: str = "eoAPI-raster"
- cors_origins: str = "*"
- cachecontrol: str = "public, max-age=3600"
- debug: bool = False
- root_path: str = ""
-
- model_config = {"env_prefix": "EOAPI_RASTER_", "env_file": ".env"}
-
- @field_validator("cors_origins")
- def parse_cors_origin(cls, v):
- """Parse CORS origins."""
- return [origin.strip() for origin in v.split(",")]
diff --git a/runtime/eoapi/raster/eoapi/raster/templates/landing.html b/runtime/eoapi/raster/eoapi/raster/templates/landing.html
deleted file mode 100644
index a71d3fa..0000000
--- a/runtime/eoapi/raster/eoapi/raster/templates/landing.html
+++ /dev/null
@@ -1,72 +0,0 @@
-
-
-
- {{ template.title }}
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- 
+ 
