From 4eade309a50f5d828652cf7b62889dfdfdfe0c2d Mon Sep 17 00:00:00 2001 From: taniya-das Date: Mon, 11 Sep 2023 11:39:28 +0200 Subject: [PATCH 1/8] update README --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index e78c6c06..6a0450d3 100644 --- a/README.md +++ b/README.md @@ -51,6 +51,7 @@ following figure: ## Prerequisites - Linux/MacOS/Windows (should all work) - [Docker](https://docs.docker.com/get-docker/) +- [Docker Compose](https://docs.docker.com/compose/install/) version 2.21.0 or higher For development: - `Python3.11` with `python3.11-dev` (`sudo apt install python3.11-dev` on Debian) From 10ff0eacd049dce32cc54e158be1d21bbd68ecb7 Mon Sep 17 00:00:00 2001 From: taniya-das Date: Tue, 31 Oct 2023 16:55:15 +0100 Subject: [PATCH 2/8] removed script/run_mysql_server.sh --- README.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/README.md b/README.md index 9efd5fb3..4217d14e 100644 --- a/README.md +++ b/README.md @@ -115,8 +115,6 @@ mysql> SHOW DATABASES; 4 rows in set (0.03 sec) ``` -For ease of use, you can also make use of the script in `scripts/run_mysql_server.sh`. - #### Persistent Storage The data is persistent when simply stopping and restarting the server: From 7bb8d37352102a4a4b5850cfc1ab653df2ec7d7d Mon Sep 17 00:00:00 2001 From: taniya-das Date: Fri, 3 Nov 2023 14:01:37 +0100 Subject: [PATCH 3/8] Modified README.md --- README.md | 68 ++----------------------------------------------------- 1 file changed, 2 insertions(+), 66 deletions(-) diff --git a/README.md b/README.md index 4217d14e..86b86d15 100644 --- a/README.md +++ b/README.md @@ -78,28 +78,11 @@ This configuration is intended for development, DO NOT use it in production. Use the following instructions if you prefer to use Docker instead of Docker Compose. -### Starting a MySQL Server - We use the default [MySQL Docker image](https://hub.docker.com/_/mysql). -By default, the database is stored within the docker container and will thus be deleted when the container is removed. -Instructions on using a persistent storage can be found at the end of this section. - -First, we define a docker network to allow our server to be reachable from other docker containers: - -```bash -docker network create sql-network -``` - -Then, start the MySQL Server: +To connect to the database ```bash -docker run -e MYSQL_ROOT_PASSWORD=ok --name sqlserver --network sql-network -d mysql -``` - -That's all! You should be able to connect to the server now, though no database is present yet: - -```bash -docker run -it --network sql-network --rm mysql mysql -hsqlserver -uroot -pok +./scripts/database-connect.sql ``` ```bash @@ -113,46 +96,7 @@ mysql> SHOW DATABASES; | sys | +--------------------+ 4 rows in set (0.03 sec) -``` - -#### Persistent Storage -The data is persistent when simply stopping and restarting the server: - -```bash -docker stop sqlserver -docker start sqlserver -``` - -However, all data is lost when the container is deleted. -To ensure data can persist even if the container is deleted, allow the Docker container to write to a directory on the -host machine. -To do that, mount a volume by adding `-v /ABSOLUTE/PATH/TO/HOST/DIR:/var/lib/mysql` to the docker command that starts -the server -(it's also possible to create a docker -volume ([docs](https://docs.docker.com/engine/reference/commandline/run/#mount-volume--v---read-only))). -If you want to use a path within this repository directory, we recommend naming the directory `data` since then it will -automatically be ignored by git. -For more information, see "Where to Store Data" in the linked documentation. - -### Starting the REST API - -The repository provides a Dockerfile to run the REST API in a containerized environment. - -#### Using the Docker container - -First, build the docker image from the dockerfile: - -```bash -docker build --tag ai4eu_server_demo:latest -f Dockerfile . -``` - -then create a container from that image, remember to forward the port and connect to the right -docker network (alternatively you can use the script at `scripts/run_apiserver.sh`) - -```bash -docker run --network sql-network -it --rm -p 8000:8000 --name apiserver ai4eu_server_demo -``` At this point you should be able to visit the server from your browser at `localhost:8000/docs`. @@ -206,9 +150,6 @@ this work, you need to set an environment variable. You can do this by setting t KEYCLOAK_CLIENT_SECRET=[SECRET] ``` -Please ask Jos van der Velde (j.d.v.d.velde@tue.nl) for the keycloak secret, and to give your -user the correct roles. - Alternatively, you can connect to a different keycloak instance by modifying `src/.env`. EGI Checkin can for instance be used on a deployed instance - not on local host. Marco Rorro is the go-to person to request the usage of the EGI Checkin. @@ -253,11 +194,6 @@ You can change this behavior through parameters of the script: Following the installation instructions above, the server may be reached at `127.0.0.1:8000`. REST API documentation is automatically built and can be viewed at `127.0.0.1:8000/docs`. -#### Editing Files Locally - -While developing the server it is often convenient to edit the files on the host machine. -To avoid rebuilding the docker container each time you edit the files, you can mount the host files into the container. -Mount your repository's `src` directory into the container's `/app` directory by adding `-v $(pwd)/src:/app`. #### Automatically Restart on Change From c95f757c0bf0033552a62ed1230d2e1780f9a2d5 Mon Sep 17 00:00:00 2001 From: taniya-das Date: Fri, 3 Nov 2023 14:40:34 +0100 Subject: [PATCH 4/8] further modifications --- README.md | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 86b86d15..9f5e464e 100644 --- a/README.md +++ b/README.md @@ -70,20 +70,14 @@ Information on how to install Docker is found in [their documentation](https://d docker compose up ``` starts the MYSQL Server, the REST API, Keycloak for Identy and access management and Nginx for reverse proxing. \ -Once started, you should be able to visit the REST API server at: http://localhost and Keycloak at http://localhost/aiod-auth \ +Once started, you should be able to visit the REST API server at: http://localhost:8000 and Keycloak at http://localhost:8080 \ To authenticate to the REST API swagger interface the predefined user is: user, and password: password \ To authenticate as admin to Keycloak the predefined user is: admin and password: password \ To use a different DNS hostname replace localhost with it in .env and src/config.toml \ This configuration is intended for development, DO NOT use it in production. -Use the following instructions if you prefer to use Docker instead of Docker Compose. - We use the default [MySQL Docker image](https://hub.docker.com/_/mysql). -To connect to the database - -```bash -./scripts/database-connect.sql -``` +To connect to the database use `./scripts/database-connect.sql`. ```bash mysql> SHOW DATABASES; @@ -97,8 +91,8 @@ mysql> SHOW DATABASES; +--------------------+ 4 rows in set (0.03 sec) +Now, you can visit the server from your browser at `localhost:8000/docs`. -At this point you should be able to visit the server from your browser at `localhost:8000/docs`. #### Local Installation From 56ab0dd43b8e45d2c6eb93d81aa1f047c763a655 Mon Sep 17 00:00:00 2001 From: taniya-das Date: Fri, 3 Nov 2023 14:41:59 +0100 Subject: [PATCH 5/8] further modifications --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 9f5e464e..7b376396 100644 --- a/README.md +++ b/README.md @@ -76,7 +76,7 @@ To authenticate as admin to Keycloak the predefined user is: admin and password: To use a different DNS hostname replace localhost with it in .env and src/config.toml \ This configuration is intended for development, DO NOT use it in production. -We use the default [MySQL Docker image](https://hub.docker.com/_/mysql). +We use the default [MySQL Docker image](https://hub.docker.com/_/mysql).\ To connect to the database use `./scripts/database-connect.sql`. ```bash From 858de62fa8362e261f5ba95578b1760e51adbd34 Mon Sep 17 00:00:00 2001 From: taniya-das Date: Fri, 3 Nov 2023 14:42:49 +0100 Subject: [PATCH 6/8] further modifications --- README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/README.md b/README.md index 7b376396..066ded02 100644 --- a/README.md +++ b/README.md @@ -76,7 +76,6 @@ To authenticate as admin to Keycloak the predefined user is: admin and password: To use a different DNS hostname replace localhost with it in .env and src/config.toml \ This configuration is intended for development, DO NOT use it in production. -We use the default [MySQL Docker image](https://hub.docker.com/_/mysql).\ To connect to the database use `./scripts/database-connect.sql`. ```bash From 1e96b867e32a918d54486478bfd4c61911c80b3e Mon Sep 17 00:00:00 2001 From: Jos van der Velde Date: Wed, 8 Nov 2023 10:49:47 +0100 Subject: [PATCH 7/8] Making sure the deletion triggers are created once --- src/main.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/main.py b/src/main.py index 5e192359..a676eb52 100644 --- a/src/main.py +++ b/src/main.py @@ -92,7 +92,6 @@ def create_app() -> FastAPI: }, ) engine = sqlmodel_engine(args.rebuild_db) - add_delete_triggers(AIoDConcept) with engine.connect() as connection: AIoDConcept.metadata.create_all(connection, checkfirst=True) connection.commit() @@ -102,6 +101,11 @@ def create_app() -> FastAPI: session.add_all([Platform(name=name) for name in PlatformName]) session.commit() + # this is a bit of a hack: instead of checking whether the triggers exist, we check + # whether platforms are already present. If platforms were not present, the db is + # empty, and so the triggers should still be added. + add_delete_triggers(AIoDConcept) + add_routes(app, engine, url_prefix=args.url_prefix) return app From be5e2611c9c67c2cca04e0c0b1685c49aab1124e Mon Sep 17 00:00:00 2001 From: taniya-das Date: Wed, 8 Nov 2023 13:43:47 +0100 Subject: [PATCH 8/8] corrections made --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 066ded02..261a1177 100644 --- a/README.md +++ b/README.md @@ -70,7 +70,7 @@ Information on how to install Docker is found in [their documentation](https://d docker compose up ``` starts the MYSQL Server, the REST API, Keycloak for Identy and access management and Nginx for reverse proxing. \ -Once started, you should be able to visit the REST API server at: http://localhost:8000 and Keycloak at http://localhost:8080 \ +Once started, you should be able to visit the REST API server at: http://localhost and Keycloak at http://localhost/aiod-auth \ To authenticate to the REST API swagger interface the predefined user is: user, and password: password \ To authenticate as admin to Keycloak the predefined user is: admin and password: password \ To use a different DNS hostname replace localhost with it in .env and src/config.toml \ @@ -89,6 +89,7 @@ mysql> SHOW DATABASES; | sys | +--------------------+ 4 rows in set (0.03 sec) +``` Now, you can visit the server from your browser at `localhost:8000/docs`. @@ -134,7 +135,7 @@ The `--reload` argument will automatically restart the app if changes are made t 3. Run using DevContainer (see next subsection) ### Authentication -Currently, the code is on default coupled with a keycloak running on test.openml.org. To make +Currently, the code is by default running using the local Keycloak. To make this work, you need to set an environment variable. You can do this by setting the `KEYCLOAK_CLIENT_SECRET` in `src/.env`.