Skip to content

Latest commit

 

History

History
199 lines (130 loc) · 9.08 KB

README.md

File metadata and controls

199 lines (130 loc) · 9.08 KB
Raw

OMOP Automatic Deployment - For Docker Compose, Kubernetes & Istio

OMOP Automatic Deploy - For Docker Compose, Kubernetes & Istio

This repository contains the necessary files to deploy the OMOP database using Kubernetes or Docker Compose, and the necessary files to populate the database with the OMOP vocabulary that you want. This repository is based on the OMOP CommonDataModel repository and uses a custom version of the OHDSI Broadsea vocabulary loader.

Disclaimer: The vocabularies that require an API Key from UML have not been tested with this deployment method.

Table of Contents

  1. Prerequisites
  2. Deployment
    1. Docker Compose
    2. Bare Kubernetes
    3. Istio Kubernetes
  3. License

Prerequisites

Deployment

Docker Compose

  1. Clone this repository:

    git clone https://gitlab.lst.tfo.upm.es/idea4rc/omop-deploy.git

  2. In docker-compose.yaml, change all args variables and environment variables to the values you want. These are the variables used:

DB service variables:

Variable name Explanation Default Value
POSTGRES_USER Name of the administration user of the database postgres
POSTGRES_PASSWORD Password of the administration user of the database mysecretpassword
POSTGRES_DB Name of the database omopdb

vocab_uploader service variables:

Variable name Explanation Default Value
VOCAB_PG_HOST Host of the database db (must have the name of the service containing the database)
VOCAB_PG_DATABASE Name of the database omopdb (must be the same as in database service)
VOCAB_PG_USER Name of the administration user of the database postgres (must be the same as in database service)
VOCAB_PG_PASSWORD Password of the administration user of the database mysecretpassword (must be the same as in database service)
VOCAB_PG_SCHEMA Schema where the vocabulary will be loaded omopschema

  1. In order to load the vocabularies when the database is ready, the vocab_loader script is used. This script uses the CSV files downloaded from OHDSI ATHENA and its part of OHDSI BROADSEA repository. To load the vocabularies, you must download the CSV files from ATHENA and put them in the vocab_script/files folder. Don't change the files names, because the script uses them to load the vocabularies.

  2. When everything is ready, you must execute the docker-compose.yaml using this command:

    docker compose up -d

The variables used and its locations are:

OMOP CDM Deployment variables:

Variable name Explanation Default Value
POSTGRES_USER Name of the administration user of the database postgres
POSTGRES_DB Name of the database omopdb

OMOP Secrets variables:

Variable name Explanation Default Value
POSTGRES_PASSWORD Password of the administration user of the database mysecretpassword (Written in Base64)

OMOP Vocab Job variables:

Variable name Explanation Default Value
VOCAB_PG_HOST Host of the database db (must have the name of the service containing the database)
VOCAB_PG_DATABASE Name of the database omopdb (must be the same as in database service)
VOCAB_PG_USER Name of the administration user of the database postgres (must be the same as in database service)
VOCAB_PG_PASSWORD Password of the administration user of the database mysecretpassword (must be the same as in database service)
VOCAB_PG_SCHEMA Schema where the vocabulary will be loaded omopcdm

THIS VARIABLES SHOULD BE PUT ON THE ARGS SECTION AND THE ENV SECTION

  1. In order to load the vocabularies when the database is ready, the vocab_loader script is used. This script uses the CSV files downloaded from OHDSI ATHENA and its part of OHDSI BROADSEA repository To load the vocabularies, you must download the CSV files from ATHENA and put them in the vocab_script/files folder. Don't change the files names, because the script uses them to load the vocabularies.

  2. When everything is ready, you must execute the kubernetes.yaml using this command:

    kubectl apply -f kubernetes/bare-kubernetes/001_OMOP-Secrets.yaml

kubectl apply -f kubernetes/bare-kubernetes/002_OMOP-CDM-Deployment.yaml

kubectl apply -f kubernetes/bare-kubernetes/003_OMOP-CDM-Service.yaml

kubectl apply -f kubernetes/bare-kubernetes/004_OMOP-Vocab-Job.yaml

Istio Kubernetes

This method is for a Kubernetes cluster with Istio installed. If you want to deploy the OMOP database in a vanilla Kubernetes cluster, you can use the Bare Kubernetes deployment method.

This tutorial deploys a new namespace called Datamesh, with a policy that only allows mTLS traffic, and then we deploy all the OMOP database components in this namespace. If you want to deploy the OMOP database in another namespace, you should change the namespace in the YAML files.

  1. Download Istio and enable the Istio injection in the cluster:

    istioctl install --set profile=demo -y

  2. Clone this repository:

    git clone https://gitlab.lst.tfo.upm.es/idea4rc/omop-deploy.git

  3. In Kubernetes you should change all args variables and environment variables to the values you want. These are displayed in this format:

    env:
  - name: VARIABLE_NAME
    value: VARIABLE_VALUE

    The variables used and its locations are:

OMOP CDM Deployment variables:

Variable name Explanation Default Value
POSTGRES_USER Name of the administration user of the database postgres
POSTGRES_DB Name of the database omopdb

OMOP Secrets variables:

Variable name Explanation Default Value
POSTGRES_PASSWORD Password of the administration user of the database mysecretpassword (Written in Base64)

OMOP Vocab Job variables:

Variable name Explanation Default Value
VOCAB_PG_HOST Host of the database db (must have the name of the service containing the database)
VOCAB_PG_DATABASE Name of the database omopdb (must be the same as in database service)
VOCAB_PG_USER Name of the administration user of the database postgres (must be the same as in database service)
VOCAB_PG_PASSWORD Password of the administration user of the database mysecretpassword (must be the same as in database service)
VOCAB_PG_SCHEMA Schema where the vocabulary will be loaded omopschema

THIS VARIABLES SHOULD BE PUT ON THE ARGS SECTION AND THE ENV SECTION

  1. In order to load the vocabularies when the database is ready, the vocab_loader script is used. This script uses the CSV files downloaded from OHDSI ATHENA and its part of OHDSI BROADSEA repository. To load the vocabularies, you must download the CSV files from ATHENA and put them in the vocab_script/files folder. Don't change the files names, because the script uses them to load the vocabularies.

  2. When everything is ready, you must execute the kubernetes.yaml using this command:

    kubectl apply -f kubernetes/istio/001_datamesh.yaml

kubectl apply -f kubernetes/datamesh/002_mtls-policy.yaml

kubectl apply -f kubernetes/istio/003_gateway.yaml

kubectl apply -f kubernetes/istio/004_omop-secrets.yaml

kubectl apply -f kubernetes/istio/005_omop-db-cdm-deploy.yaml

kubectl apply -f kubernetes/istio/006_omop-db-svc-datamesh.yaml

kubectl apply -f kubernetes/istio/007_omop-vocab-job.yaml

License

This software is licensed under Apache License 2.0. This license applies for this file and the other files hosted in this repository.

Copyright 2022 Universidad Politécnica de Madrid

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

See LICENSE for details.