Updates documentation (#927)

- Improves developer documentation for client - Refactors documentation for docker scenarios - Adds user and example videos - Adds new run-time monitoring example
INTO-CPS-Association · Sep 21, 2024 · cede74b · cede74b
1 parent 14fbd92
commit cede74b
Show file tree

Hide file tree

Showing 18 changed files with 385 additions and 171 deletions.
diff --git a/client/DEVELOPER.md b/client/DEVELOPER.md
@@ -88,6 +88,16 @@ It is therefore reccommend to keep the configurations in the
 `client/config/` directory and use one of the `yarn config`
 sub-commands to switch between them.
 
+The purpose of different configuration files are given in the table below.
+
+| Configuration File Name | Usage Scenario |
+|:--------|:--------|
+| `dev.js` | Check functionality of the client application without using any other parts of the DTaaS software |
+| `gitlab.js` | Used for testing the gitlab code written in `src/utils/gitlab.ts`. These credentials are not used by the client application |
+| `local.js` | Used by the `docker/compose.dev.yml` to setup a check the complete DTaaS application on the developer computer |
+| `prod.js` | Used for running the client application on a production server. This file is kept here for reference. The `deploy/config/client/env.js` is used during installations. |
+| `test.js` | Used by all the tests |
+
 ## Example configuration for developers
 
 The first step is to collect the URL of gitlab acting as oauth provider.

diff --git a/client/config/dev.js b/client/config/dev.js
@@ -1,7 +1,7 @@
 if (typeof window !== 'undefined') {
   window.env = {
     REACT_APP_ENVIRONMENT: 'dev',
-    REACT_APP_URL: 'http://localhost/',
+    REACT_APP_URL: 'http://localhost:4000/',
     REACT_APP_URL_BASENAME: '',
     REACT_APP_URL_DTLINK: '/lab',
     REACT_APP_URL_LIBLINK: '',
@@ -12,8 +12,8 @@ if (typeof window !== 'undefined') {
 
     REACT_APP_CLIENT_ID: '1be55736756190b3ace4c2c4fb19bde386d1dcc748d20b47ea8cfb5935b8446c',
     REACT_APP_AUTH_AUTHORITY: 'https://gitlab.foo.com/',
-    REACT_APP_REDIRECT_URI: 'http://localhost/Library',
-    REACT_APP_LOGOUT_REDIRECT_URI: 'http://localhost/',
+    REACT_APP_REDIRECT_URI: 'http://localhost:4000/Library',
+    REACT_APP_LOGOUT_REDIRECT_URI: 'http://localhost:4000',
     REACT_APP_GITLAB_SCOPES: 'openid profile read_user read_repository api',
   };
 };
diff --git a/client/package.json b/client/package.json
@@ -16,7 +16,7 @@
   "type": "module",
   "scripts": {
     "build": "npx shx cp config/gitlab.json src/util/gitlab.json && npx react-scripts build && npx rimraf src/util/gitlab.json",
-    "clean": "npx rimraf build/ dist/ node_modules/ coverage/ playwright-report/  *.svg src/util/gitlab.json",
+    "clean": "npx rimraf build/ dist/ node_modules/ coverage/ playwright-report/ test-results/  *.svg src/util/gitlab.json",
     "config:dev": "npx shx cp config/dev.js public/env.js && npx shx cp config/dev.js build/env.js",
     "config:local": "npx shx cp config/local.js public/env.js && npx shx cp config/local.js build/env.js",
     "config:prod": "npx shx cp config/prod.js public/env.js && npx shx cp config/prod.js build/env.js",

diff --git a/client/test/README.md b/client/test/README.md
@@ -1,42 +1,25 @@
 # End-to-End (E2E) Tests
 
-The E2E tests require playwright test runner, an on-premise GitLab OAuth setup and
-configured `config/test.js` and `test/.env` files. When everything is set up, you
-can run the tests by running `yarn test:e2e`.
+The E2E tests require playwright test runner and an on-premise GitLab OAuth setup.
+The <https://gitlab.com> has CAPTCHA protection blocking the end-to-end tests.
+Thus on-premise or third-party Gitlab instance hosted without CAPTCHA protection
+is needed.
 
-It is also possible to test the hosted DTaaS applications hosted at a URL,
-say `https://foo.com` using `yarn test:e2e:ext`. Remember to set the environment
-variable in `test/.env` to the URL of the hosted DTaaS application. An example
-is shown below:
+Make sure you have an active internet connection while running these tests,
+as the tests simulate real user interactions with your GitLab account.
 
-```js
-if (typeof window !== 'undefined') {
-  window.env = {
-    REACT_APP_ENVIRONMENT: 'test',
-    REACT_APP_URL: 'https://foo.com/',
-    REACT_APP_URL_BASENAME: '',
-    REACT_APP_URL_DTLINK: '/lab',
-    REACT_APP_URL_LIBLINK: '',
-    REACT_APP_WORKBENCHLINK_VNCDESKTOP: '/tools/vnc/?password=vncpassword',
-    REACT_APP_WORKBENCHLINK_VSCODE: '/tools/vscode/',
-    REACT_APP_WORKBENCHLINK_JUPYTERLAB: '/lab',
-    REACT_APP_WORKBENCHLINK_JUPYTERNOTEBOOK: '',
-
-    REACT_APP_CLIENT_ID: '1be55736756190b3ace4c2c4fb19bde386d1dcc748d20b47ea8cfb5935b8446c',
-    REACT_APP_AUTH_AUTHORITY: 'https://gitlab.com/',
-    REACT_APP_REDIRECT_URI: 'https://foo.com/Library',
-    REACT_APP_LOGOUT_REDIRECT_URI: 'https://foo.com/',
-    REACT_APP_GITLAB_SCOPES: 'openid profile read_user read_repository api',
-  };
-};
-```
+There are two possible testing setups you can create.
+
+1. Host website on the developer computer and test from developer computer.
+   This is the default E2E testing scenario.
+   The DTaaS client application will be running at `http://localhost:4000`.
+1. Host website on the integration server and test from the integration server.
+   The DTaaS client application will be running at `https://foo.com`.
 
-The `yarn install` and `yarn config:test` need to be run before `yarn test:e2e:ext`
-can be run successfully. Also note that if you are deploying the client
-application with Traeffik forward authorization, the tests will fail due to
-the additionally required Gitlab authorization.
+The following sections describe configuration and yarn test commands for
+both these scenarios.
 
-## Playwright
+## Install Playwright
 
 The E2E tests use playwright test runner. You also need to have the software
 installed. If it is not installed, you can install it with the following command.
@@ -45,21 +28,31 @@ installed. If it is not installed, you can install it with the following command
 yarn playwright install --with-deps
 ```
 
-## OAuth Setup
+## Setup Test Configuration
 
-You can follow the instructions in [authorization page
-](../../docs/admin/client/auth.md)to setup OAuth for the react client website.
-Remember to add the `http://localhost:4000` as callback URL in the OAuth
-application. The GitLab will still be running on a remote machine.
+### OAuth Setup
+
+You can follow the instructions in
+[authorization page](../../docs/admin/client/auth.md) to setup OAuth for
+the react client website.
+Remember to add the correct callback URL in the OAuth
+application. Depending the location of the client website,
+one of the following URLs must be registered as callback URL.
+
+| Location of client application | URL |
+|:---|:---|
+| Localhost | `http://localhost:4000` |
+| External / Integration server | `https://foo.com` |
+
+The GitLab will still be running on a remote machine.
 It is not possible to run both the GitLab and react client website on localhost.
 
-## config/test.js file
+### Client Configuration
 
-Before running the E2E tests, you need to update the `config/test.js` file.
-If you have a live DTaaS client website running, you can copy the `build/env.js`
-into `config/test.js`.
+Before running the E2E tests, you need to update
+the client configuration file available at `config/test.js`.
 
-Open `config/test.js` in a text editor and make sure the configuration matches
+Make sure the configuration in `config/test.js` matches
 the details of your testing environment. For instance, you need to adjust:
 
 * `REACT_APP_URL`
@@ -69,66 +62,10 @@ the details of your testing environment. For instance, you need to adjust:
 
 to reflect your test setup. More information on about the environment settings is
 available in [authorization](../../docs/admin/client/auth.md) and
-[client deployment](../../docs/admin/client/CLIENT.md) pages.
+[client configuration](../../docs/admin/client/config.md) pages.
 
 Here's an example of relevant values for variables. This example is suitable for
-testing on developer computer.
-
-```js
-REACT_APP_URL="http://localhost:4000"
-REACT_APP_AUTH_AUTHORITY="http://gitlab.foo.com"
-REACT_APP_REDIRECT_URI="http://localhost:4000/Library"
-REACT_APP_LOGOUT_REDIRECT_URI="http://localhost:4000"
-```
-
-Finally, run `yarn config:test` to copy the config file into the `build`
-and `public` folders.
-
-## env file
-
-You need to create a `test/.env` file where you will store the GitLab user
-credentials and application URL for the website. The credentials will be
-used by playwright to simulate real user interactions during the E2E tests.
-
-A template for `test/.env` is given here:
-
-```env
-REACT_APP_TEST_USERNAME=your_username
-REACT_APP_TEST_PASSWORD=your_password
-REACT_APP_URL='https://foo.com'
-```
-
-Replace _your_username_ and _your_password_ with the actual username and password
-of your GitLab account or the testing account that you intend to use. Finally
-replace _foo.com_ with the URL of your application, as you did in `env.js`.
-
-Here's an example for test setup on the developer machine and on the
-integration server:
-
-```env
-REACT_APP_TEST_USERNAME=TestUsername
-REACT_APP_TEST_PASSWORD=TestPassword123
-REACT_APP_URL='http://localhost:4000'
-```
-
-## Testing on localhost
-
-If you want to handle starting the react client server yourself, there are two
-possible testing setups you can create.
-
-1. Host website on the developer computer and test from developer computer
-1. Host website on the integration server and test from the integration server
-
-If you use `localhost` in the `REACT_APP_URL` the above the two mentioned setups
-are essentially the same.
-In order to run the tests on the integration server, you need to disable the
-HTTP authorization (if setup in the first place) on the Traefik server and
-let the website be accessible without any authenticaiton.
-
-### The configuration files for the test on localhost
-
-The `config/test.js` file is given below. The `build/env.js` also holds the
-same content.
+testing on developer computer, i.e., `localhost`.
 
 ```js
 window.env = {
@@ -150,26 +87,8 @@ window.env = {
 };
 ```
 
-## test/.env - localhost
-
-```ini
-REACT_APP_TEST_USERNAME=TestUsername
-REACT_APP_TEST_PASSWORD=TestPassword123
-REACT_APP_URL='http://localhost:4000'
-```
-
-Please note that the username and password are the user
-credentials on `gitlab.foo.com`.
-
-## Testing on the integration server
-
-In this test setup, the DTaaS application is running at `https://foo.com` and
-the GitLab instance is running at `https://gitlab.foo.com`. The E2E test shall
-be run from the developer computer. The codebase commit should be the same on
-both the developer computer and integration server.
-
-The `config/test.js` file on the developer computer is given below. The
-`build/env.js` of the integration server also holds the same content.
+The matching values for the running the DTaaS client applicat on an integration
+server hosted at `https://foo.com` are:
 
 ```js
 window.env = {
@@ -191,38 +110,76 @@ window.env = {
 };
 ```
 
-## test/.env - Integration server
+### Test User Credentials
+
+You need to create a test environment file named `test/.env`
+in which you will store the GitLab user credentials and
+the DTaaS application URL for the website. The credentials will be
+used by playwright to simulate real user interactions during the E2E tests.
+
+A template for `test/.env` for running the DTaaS client application
+on the developer computer, i.e., `localhost` is given here:
+
+```env
+REACT_APP_TEST_USERNAME=your_username
+REACT_APP_TEST_PASSWORD=your_password
+REACT_APP_URL='http://localhost:4000'
+```
+
+Replace _your_username_ and _your_password_ with the actual username and password
+of your on-premise GitLab account (`gitlab.foo.com`) or the testing account that
+you intend to use.
+
+Here's an example `test/.env` for test setup on the developer machine and
+and the DTaaS client application running on a remote integration server:
 
-```ini
+```env
 REACT_APP_TEST_USERNAME=TestUsername
 REACT_APP_TEST_PASSWORD=TestPassword123
 REACT_APP_URL='https://foo.com'
 ```
 
-Please note that the username and password are the user credentials on `gitlab.foo.com`.
+Here `https://foo.com` is the URL of the application.
+Remember to replace _foo.com_ with the URL of your application.
 
-**NOTE:** The tests from developer computer to the integration server only
-work with null basename. The test fails if a basename (say `au`) is specified.
-This might be due to a complex interaction of developer computer, traefik
-gateway and the client website hosted behind traefik.
+## Run Tests
 
-## Running the Tests
+### Localhost
 
-Once you've properly set up your .env file, you can run the end-to-end tests as follows:
+You can run the end-to-end tests as follows:
 
 ```bash
+yarn install
+yarn build
+yarn config:test
 yarn test:e2e
 ```
 
-Or with manual website launch:
+The `yarn test:e2e` command launches the test runner, the DTaaS client application
+and execute all end-to-end tests.
+The client application is terminated at the end of end-to-end tests.
+
+## Testing on the integration server
+
+In this test setup, the DTaaS application is running at `https://foo.com` and
+the GitLab instance is running at `https://gitlab.foo.com`. The E2E test shall
+be run from the developer computer. The codebase commit should be the same on
+both the developer computer and integration server.
+
+Points to note:
+
+1. In order to run the tests on the integration server, you need to disable the
+   HTTPS authorization (if setup in the first place) on the Traefik server and
+   let the website be accessible without any authenticaiton done by
+   [Traefik forward auth](../../docs/admin/servers/auth.md) service.
+1. The tests from developer computer to the integration server only
+   work with null basename. The tests fail if a basename (say `au`) is specified.
+   This might be due to a complex interaction of developer computer, traefik
+   gateway and the client website hosted behind traefik.
+
+You can run the end-to-end tests as follows:
 
 ```bash
+yarn install
 yarn test:e2e:ext
 ```
-
-These commands launch the test runner and execute all end-to-end tests. The first
-command also runs the `yarn start` command to start the client website
-and terminates it after testing.
-
-Make sure you have an active internet connection while running these tests,
-as they simulate real user interactions with your GitLab account.
diff --git a/deploy/docker/SERVER.md b/deploy/docker/SERVER.md
@@ -269,7 +269,7 @@ docker compose -f compose.server.secure.yml --env-file .env.server up -d --force
 ## Use
 
 The application will be accessible at:
-<http(s)://foo.com> from web browser.
+`http(s)://foo.com` from web browser.
 Sign in using your account linked to
 either _gitlab.com_ or your local gitlab instance.
 

diff --git a/deploy/services/README.md b/deploy/services/README.md
@@ -9,6 +9,7 @@ The installation scripts in this directory install:
 * **Grafana** visualization and dashboard service
 * **RabbitMQ** AMQP broker and its' management interface
 * Eclipse Mosquitto **MQTT** broker
+* **MongoDB** database server
 
 ## Configure and Install
 
@@ -35,7 +36,7 @@ at the following ports / URLs.
 | RabbitMQ Broker | services.foo.com:5672 |
 | RabbitMQ Broker Management Website | services.foo.com:15672 |
 | MQTT Broker | services.foo.com:1883 |
-||
+| MongoDB database | services.foo.com:27017 |
 
 The firewall and network access settings of corporate / cloud network need to be
 configured to allow external access to the services. Otherwise the users of DTaaS

diff --git a/docker/README.md b/docker/README.md
@@ -66,7 +66,7 @@ The configuration files to be updated are:
 1. docker/conf.dev
    please see [server docs](../docs/admin/servers/auth.md) for help
    with updating this config file)
-1. client/config/dev.js
+1. client/config/local.js
    please see [client config](../docs/admin/client/config.md) for help
    with updating this config file)
 

diff --git a/docs/admin/client/config.md b/docs/admin/client/config.md
@@ -49,7 +49,7 @@ This page describes various configuration options for react website.
   if (typeof window !== 'undefined') {
     window.env = {
       REACT_APP_ENVIRONMENT: "dev",
-      REACT_APP_URL: 'https://foo.com/',
+      REACT_APP_URL: 'http://localhost:4000/',
       REACT_APP_URL_BASENAME: 'bar',
       REACT_APP_URL_DTLINK: '/lab',
       REACT_APP_URL_LIBLINK: '',
@@ -59,8 +59,8 @@ This page describes various configuration options for react website.
       REACT_APP_WORKBENCHLINK_JUPYTERNOTEBOOK: '',
       REACT_APP_CLIENT_ID: '1be55736756190b3ace4c2c4fb19bde386d1dcc748d20b47ea8cfb5935b8446c',
       REACT_APP_AUTH_AUTHORITY: 'https://gitlab.foo.com/',
-      REACT_APP_REDIRECT_URI: 'https://foo.com/bar/Library',
-      REACT_APP_LOGOUT_REDIRECT_URI: 'https://foo.com/bar',
+      REACT_APP_REDIRECT_URI: 'http://localhost:4000/bar/Library',
+      REACT_APP_LOGOUT_REDIRECT_URI: 'http://localhost:4000/bar',
       REACT_APP_GITLAB_SCOPES: 'openid profile read_user read_repository api',
     };
   };

diff --git a/docs/developer/index.md b/docs/developer/index.md
@@ -1,4 +1,4 @@
-# Developers Guide
+# Developer Guide
 
 This guide is for DTaaS platform developers.
 It helps development team members get familiar with