add: StreamSage

[singjc]

This PR adds StreamSage built with the streamlit-template.

@t0mdavid-m, @Arslan-Siraj, when I try to build and launch all the services with docker compose up --build -d, it seems to fail when cloning OpenMS for one of the apps. I have tried to re-run the command several times, because I thought maybe there is an issue with the network when cloning, but I still get the same error.

 => ERROR [nuxl-app setup-build-system 17/19] RUN git clone --recursive --depth=1 -b feature/NuXL --single-  81.6s

I'm not sure if it's due to a different version of docker compose that I'm using. Have you come across this or know what the issue might be?

Version Info

$ lsb_release -a
Description:	Pop!_OS 22.04 LTS
Release:	22.04
$ docker compose version
Docker Compose version v2.32.1

Full log

$ docker compose up --build -d
WARN[0000] The "GITHUB_TOKEN" variable is not set. Defaulting to a blank string. 
WARN[0000] The "GITHUB_TOKEN" variable is not set. Defaulting to a blank string. 
WARN[0000] The "GITHUB_TOKEN" variable is not set. Defaulting to a blank string. 
WARN[0000] The "GITHUB_TOKEN" variable is not set. Defaulting to a blank string. 
WARN[0000] The "GITHUB_TOKEN" variable is not set. Defaulting to a blank string. 
WARN[0000] The "GITHUB_TOKEN" variable is not set. Defaulting to a blank string. 
WARN[0000] The "GITHUB_TOKEN" variable is not set. Defaulting to a blank string. 
WARN[0000] /home/singjc/Documents/github/streamlit-deployment/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion 
[+] Running 0/0
[+] Running 0/7weis                   Building                                                                0.1s 
 ⠙ Service naseweis                   Building                                                                0.2s 
 ⠙ Service umetaflow                  Building                                                                0.2s 
 ⠙ Service stream-sage                Building                                                                0.2s 
 ⠙ Service PTMScanner                 Building                                                                0.2s 
 ⠙ Service openms-streamlit-template  Building                                                                0.2s 
 ⠙ Service nuxl-app                   Building                                                                0.2s 
 ⠦ Service umetaflow                  Building                                                               74.7s 
[+] Building 74.6s (126/390)                                                                        docker:default
 ⠦ Service stream-sage                Building                                                               74.7s 
 => [umetaflow setup-build-system 15/17] RUN git clone --recursive --depth=1 -b develop --single-branch htt  73.9s
 ⠦ Service PTMScanner                 Building                                                               74.7s 
 => [naseweis setup-build-system 15/17] RUN git clone --recursive --depth=1 -b develop --single-branch http  73.9s
 ⠇ Service stream-sage                Building                                                               76.9s 
 ⠇ Service PTMScanner                 Building                                                               76.9s 
 ⠇ Service openms-streamlit-template  Building                                                               76.9s 
[+] Building 76.8s (126/390)                                                                        docker:default
 ⠇ Service nuxl-app                   Building                                                               76.9s 
[+] Building 82.5s (133/390)                                                                        docker:default 
 => [openms-streamlit-template internal] load build definition from Dockerfile                                0.0s 
 => => transferring dockerfile: 7.58kB                                                                        0.0s 
 => [umetaflow internal] load build definition from Dockerfile                                                0.0s 
 => => transferring dockerfile: 7.14kB                                                                        0.0s 
 => [FLASHViewer internal] load build definition from Dockerfile                                              0.0s 
 => => transferring dockerfile: 8.01kB                                                                        0.0s 
 => [nuxl-app internal] load build definition from Dockerfile                                                 0.0s 
 => => transferring dockerfile: 8.83kB                                                                        0.0s 
 => [PTMScanner internal] load build definition from Dockerfile                                               0.0s 
 => => transferring dockerfile: 7.16kB                                                                        0.0s 
 => [naseweis internal] load build definition from Dockerfile                                                 0.0s 
 => => transferring dockerfile: 7.17kB                                                                        0.0s 
 => [stream-sage internal] load build definition from Dockerfile                                              0.0s 
 => => transferring dockerfile: 7.92kB                                                                        0.0s 
 => [nuxl-app internal] load metadata for docker.io/library/ubuntu:22.04                                      0.6s 
 => [FLASHViewer internal] load metadata for docker.io/library/node:21                                        0.6s 
 => [FLASHViewer auth] library/node:pull token for registry-1.docker.io                                       0.0s 
 => [openms-streamlit-template auth] library/ubuntu:pull token for registry-1.docker.io                       0.0s
 => [umetaflow internal] load .dockerignore                                                                   0.0s
 => => transferring context: 2B                                                                               0.0s
 => [openms-streamlit-template internal] load .dockerignore                                                   0.0s
 => => transferring context: 2B                                                                               0.0s
 => [naseweis internal] load .dockerignore                                                                    0.0s
 => => transferring context: 2B                                                                               0.0s
 => [nuxl-app internal] load .dockerignore                                                                    0.0s
 => => transferring context: 2B                                                                               0.0s
 => [stream-sage internal] load .dockerignore                                                                 0.0s
 => => transferring context: 2B                                                                               0.0s
 => [PTMScanner internal] load .dockerignore                                                                  0.0s
 => => transferring context: 2B                                                                               0.0s
 => [FLASHViewer internal] load .dockerignore                                                                 0.0s
 => => transferring context: 2B                                                                               0.0s
 => [umetaflow internal] load build context                                                                   0.0s
 => => transferring context: 4.04kB                                                                           0.0s
 => [FLASHViewer setup-build-system  1/17] FROM docker.io/library/ubuntu:22.04@sha256:0e5e4a57c2499249aafc3b  0.0s
 => [naseweis internal] load build context                                                                    0.0s
 => => transferring context: 2.40kB                                                                           0.0s
 => [openms-streamlit-template internal] load build context                                                   0.0s
 => => transferring context: 3.17kB                                                                           0.0s
 => [FLASHViewer internal] load build context                                                                 0.0s
 => => transferring context: 4.59kB                                                                           0.0s
 => [nuxl-app internal] load build context                                                                    0.0s
 => => transferring context: 1.95kB                                                                           0.0s
 => [stream-sage internal] load build context                                                                 0.0s
 => => transferring context: 3.69kB                                                                           0.0s
 => [PTMScanner internal] load build context                                                                  0.0s
 => => transferring context: 3.09kB                                                                           0.0s
 => [FLASHViewer js-build 1/6] FROM docker.io/library/node:21@sha256:4b232062fa976e3a966c49e9b6279efa56c8d20  0.0s
 => [FLASHViewer js-build 2/6] ADD https://api.github.com/repos/t0mdavid-m/openms-streamlit-vue-component/gi  0.2s
 => CACHED [umetaflow setup-build-system  2/17] RUN apt-get -y update                                         0.0s
 => CACHED [umetaflow setup-build-system  3/17] RUN apt-get install -y --no-install-recommends --no-install-  0.0s
 => CACHED [umetaflow setup-build-system  4/17] RUN update-ca-certificates                                    0.0s
 => CACHED [umetaflow setup-build-system  5/17] RUN apt-get install -y --no-install-recommends --no-install-  0.0s
 => CACHED [umetaflow setup-build-system  6/17] RUN apt-get install -y --no-install-recommends --no-install-  0.0s
 => CACHED [umetaflow setup-build-system  7/17] RUN apt-get install -y --no-install-recommends --no-install-  0.0s
 => CACHED [umetaflow setup-build-system  8/17] RUN wget -q     https://github.com/conda-forge/miniforge/rel  0.0s
 => CACHED [umetaflow setup-build-system  9/17] RUN mamba --version                                           0.0s
 => CACHED [umetaflow setup-build-system 10/17] COPY environment.yml ./environment.yml                        0.0s
 => CACHED [umetaflow setup-build-system 11/17] RUN mamba env create -f environment.yml                       0.0s
 => CACHED [umetaflow setup-build-system 12/17] RUN echo "mamba activate streamlit-env" >> ~/.bashrc          0.0s
 => CACHED [umetaflow setup-build-system 13/17] RUN mamba install cmake                                       0.0s
 => CACHED [umetaflow setup-build-system 14/17] RUN pip install --upgrade pip && python -m pip install -U se  0.0s
 => CACHED [naseweis setup-build-system  2/17] RUN apt-get -y update                                          0.0s
 => CACHED [naseweis setup-build-system  3/17] RUN apt-get install -y --no-install-recommends --no-install-s  0.0s
 => CACHED [naseweis setup-build-system  4/17] RUN update-ca-certificates                                     0.0s
 => CACHED [naseweis setup-build-system  5/17] RUN apt-get install -y --no-install-recommends --no-install-s  0.0s
 => CACHED [naseweis setup-build-system  6/17] RUN apt-get install -y --no-install-recommends --no-install-s  0.0s
 => CACHED [naseweis setup-build-system  7/17] RUN apt-get install -y --no-install-recommends --no-install-s  0.0s
 => CACHED [naseweis setup-build-system  8/17] RUN wget -q     https://github.com/conda-forge/miniforge/rele  0.0s
 => CACHED [naseweis setup-build-system  9/17] RUN mamba --version                                            0.0s
 => CACHED [naseweis setup-build-system 10/17] COPY environment.yml ./environment.yml                         0.0s
 => CACHED [naseweis setup-build-system 11/17] RUN mamba env create -f environment.yml                        0.0s
 => CACHED [naseweis setup-build-system 12/17] RUN echo "mamba activate streamlit-env" >> ~/.bashrc           0.0s
 => CACHED [naseweis setup-build-system 13/17] RUN mamba install cmake                                        0.0s
 => CACHED [naseweis setup-build-system 14/17] RUN pip install --upgrade pip && python -m pip install -U set  0.0s
 => CANCELED [umetaflow setup-build-system 15/17] RUN git clone --recursive --depth=1 -b develop --single-b  81.8s
 => CANCELED [naseweis setup-build-system 15/17] RUN git clone --recursive --depth=1 -b develop --single-br  81.8s
 => CACHED [stream-sage setup-build-system  2/16] RUN apt-get -y update                                       0.0s
 => CACHED [stream-sage setup-build-system  3/16] RUN apt-get install -y --no-install-recommends --no-instal  0.0s
 => CACHED [stream-sage setup-build-system  4/16] RUN update-ca-certificates                                  0.0s
 => CACHED [stream-sage setup-build-system  5/16] RUN apt-get install -y --no-install-recommends --no-instal  0.0s
 => CACHED [stream-sage setup-build-system  6/16] RUN apt-get install -y --no-install-recommends --no-instal  0.0s
 => CACHED [stream-sage setup-build-system  7/16] RUN apt-get install -y --no-install-recommends --no-instal  0.0s
 => CACHED [stream-sage setup-build-system  8/16] RUN (type -p wget >/dev/null || (apt-get update && apt-get  0.0s
 => CACHED [stream-sage setup-build-system  9/16] RUN wget -q     https://github.com/conda-forge/miniforge/r  0.0s
 => CACHED [stream-sage setup-build-system 10/16] RUN mamba --version                                         0.0s
 => CACHED [stream-sage setup-build-system 11/16] COPY environment.yml ./environment.yml                      0.0s
 => CACHED [stream-sage setup-build-system 12/16] RUN mamba env create -f environment.yml                     0.0s
 => CACHED [stream-sage setup-build-system 13/16] RUN echo "mamba activate streamlit-env" >> ~/.bashrc        0.0s
 => CACHED [stream-sage setup-build-system 14/16] RUN mamba install cmake                                     0.0s
 => CACHED [stream-sage setup-build-system 15/16] RUN pip install --upgrade pip && python -m pip install -U   0.0s
 => CANCELED [stream-sage setup-build-system 16/16] RUN git clone --recursive --depth=1 -b release/3.2.0 --  81.8s
 => CACHED [nuxl-app setup-build-system  2/19] RUN apt-get -y update                                          0.0s
 => CACHED [nuxl-app setup-build-system  3/19] RUN apt-get install -y --no-install-recommends --no-install-s  0.0s
 => CACHED [nuxl-app setup-build-system  4/19] RUN update-ca-certificates                                     0.0s
 => CACHED [nuxl-app setup-build-system  5/19] RUN apt-get install -y --no-install-recommends --no-install-s  0.0s
 => CACHED [nuxl-app setup-build-system  6/19] RUN apt-get install -y --no-install-recommends --no-install-s  0.0s
 => CACHED [nuxl-app setup-build-system  7/19] RUN apt-get install -y --no-install-recommends --no-install-s  0.0s
 => CACHED [nuxl-app setup-build-system  8/19] RUN wget -q     https://github.com/conda-forge/miniforge/rele  0.0s
 => CACHED [nuxl-app setup-build-system  9/19] RUN mamba --version                                            0.0s
 => CACHED [nuxl-app setup-build-system 10/19] COPY environment.yml ./environment.yml                         0.0s
 => CACHED [nuxl-app setup-build-system 11/19] RUN mamba env create -f environment.yml                        0.0s
 => CACHED [nuxl-app setup-build-system 12/19] RUN echo "mamba activate streamlit-env" >> ~/.bashrc           0.0s
 => CACHED [nuxl-app setup-build-system 13/19] RUN mamba install cmake                                        0.0s
 => CACHED [nuxl-app setup-build-system 14/19] RUN pip install --upgrade pip && python -m pip install -U set  0.0s
 => CACHED [nuxl-app setup-build-system 15/19] RUN git config --global http.postBuffer 524288000              0.0s
 => CACHED [nuxl-app setup-build-system 16/19] RUN git config --global http.lowSpeedTime 999999               0.0s
 => CACHED [openms-streamlit-template setup-build-system  2/18] RUN apt-get -y update                         0.0s
 => CACHED [openms-streamlit-template setup-build-system  3/18] RUN apt-get install -y --no-install-recommen  0.0s
 => CACHED [openms-streamlit-template setup-build-system  4/18] RUN update-ca-certificates                    0.0s
 => CACHED [openms-streamlit-template setup-build-system  5/18] RUN apt-get install -y --no-install-recommen  0.0s
 => CACHED [openms-streamlit-template setup-build-system  6/18] RUN apt-get install -y --no-install-recommen  0.0s
 => CACHED [openms-streamlit-template setup-build-system  7/18] RUN apt-get install -y --no-install-recommen  0.0s
 => CACHED [openms-streamlit-template setup-build-system  8/18] RUN (type -p wget >/dev/null || (apt-get upd  0.0s
 => CACHED [openms-streamlit-template setup-build-system  9/18] RUN wget -q     https://github.com/conda-for  0.0s
 => CACHED [openms-streamlit-template setup-build-system 10/18] RUN mamba --version                           0.0s
 => CACHED [openms-streamlit-template setup-build-system 11/18] COPY environment.yml ./environment.yml        0.0s
 => CACHED [openms-streamlit-template setup-build-system 12/18] RUN mamba env create -f environment.yml       0.0s
 => CACHED [openms-streamlit-template setup-build-system 13/18] RUN echo "mamba activate streamlit-env" >> ~  0.0s
 => CACHED [openms-streamlit-template setup-build-system 14/18] RUN mamba install cmake                       0.0s
 => CACHED [openms-streamlit-template setup-build-system 15/18] RUN pip install --upgrade pip && python -m p  0.0s
 => CANCELED [openms-streamlit-template setup-build-system 16/18] RUN git clone --recursive --depth=1 -b re  81.7s
 => CACHED [PTMScanner setup-build-system  2/17] RUN apt-get -y update                                        0.0s
 => CACHED [PTMScanner setup-build-system  3/17] RUN apt-get install -y --no-install-recommends --no-install  0.0s
 => CACHED [PTMScanner setup-build-system  4/17] RUN update-ca-certificates                                   0.0s
 => CACHED [PTMScanner setup-build-system  5/17] RUN apt-get install -y --no-install-recommends --no-install  0.0s
 => CACHED [PTMScanner setup-build-system  6/17] RUN apt-get install -y --no-install-recommends --no-install  0.0s
 => CACHED [PTMScanner setup-build-system  7/17] RUN apt-get install -y --no-install-recommends --no-install  0.0s
 => CACHED [PTMScanner setup-build-system  8/17] RUN wget -q     https://github.com/conda-forge/miniforge/re  0.0s
 => CACHED [PTMScanner setup-build-system  9/17] RUN mamba --version                                          0.0s
 => CACHED [PTMScanner setup-build-system 10/17] COPY environment.yml ./environment.yml                       0.0s
 => CACHED [PTMScanner setup-build-system 11/17] RUN mamba env create -f environment.yml                      0.0s
 => CACHED [PTMScanner setup-build-system 12/17] RUN echo "mamba activate streamlit-env" >> ~/.bashrc         0.0s
 => CACHED [PTMScanner setup-build-system 13/17] RUN mamba install cmake                                      0.0s
 => CACHED [PTMScanner setup-build-system 14/17] RUN pip install --upgrade pip && python -m pip install -U s  0.0s
 => ERROR [nuxl-app setup-build-system 17/19] RUN git clone --recursive --depth=1 -b feature/NuXL --single-  81.6s
 => CANCELED [PTMScanner setup-build-system 15/17] RUN git clone --recursive --depth=1 -b release/3.2.0 --s  81.7s
 => CACHED [FLASHViewer js-build 2/6] ADD https://api.github.com/repos/t0mdavid-m/openms-streamlit-vue-compo  0.0s
 => CACHED [FLASHViewer js-build 3/6] RUN git clone -b FVdeploy --single-branch https://github.com/t0mdavid-  0.0s
 => CACHED [FLASHViewer js-build 4/6] WORKDIR /openms-streamlit-vue-component                                 0.0s
 => CACHED [FLASHViewer js-build 5/6] RUN npm install                                                         0.0s
 => CACHED [FLASHViewer js-build 6/6] RUN npm run build                                                       0.0s
 => CACHED [FLASHViewer setup-build-system  2/20] COPY --from=js-build openms-streamlit-vue-component/dist /  0.0s
 => CACHED [FLASHViewer setup-build-system  3/20] RUN apt-get -y update                                       0.0s
 => CACHED [FLASHViewer setup-build-system  4/20] RUN apt-get install -y --no-install-recommends --no-instal  0.0s
 => CACHED [FLASHViewer setup-build-system  5/20] RUN update-ca-certificates                                  0.0s
 => CACHED [FLASHViewer setup-build-system  6/20] RUN apt-get install -y --no-install-recommends --no-instal  0.0s
 => CACHED [FLASHViewer setup-build-system  7/20] RUN apt-get install -y --no-install-recommends --no-instal  0.0s
 => CACHED [FLASHViewer setup-build-system  8/20] RUN apt-get install -y --no-install-recommends --no-instal  0.0s
 => CACHED [FLASHViewer setup-build-system  9/20] RUN (type -p wget >/dev/null || (apt-get update && apt-get  0.0s
 => CACHED [FLASHViewer setup-build-system 10/20] RUN wget -q     https://github.com/conda-forge/miniforge/r  0.0s
 => CACHED [FLASHViewer setup-build-system 11/20] RUN mamba --version                                         0.0s
 => CACHED [FLASHViewer setup-build-system 12/20] COPY environment.yml ./environment.yml                      0.0s
 => CANCELED [FLASHViewer setup-build-system 13/20] RUN mamba env create -f environment.yml                  81.7s
------
 > [nuxl-app setup-build-system 17/19] RUN git clone --recursive --depth=1 -b feature/NuXL --single-branch https://github.com/Arslan-Siraj/OpenMS.git && cd /OpenMS:
81.52 Cloning into 'OpenMS'...
81.52 error: RPC failed; curl 92 HTTP/2 stream 0 was not closed cleanly: CANCEL (err 8)
[+] Running 0/702 bytes of body are still expected
 ⠦ Service naseweis                   Building                                                               82.6s 
 ⠦ Service umetaflow                  Building                                                               82.6s 
 ⠦ Service stream-sage                Building                                                               82.6s 
 ⠦ Service PTMScanner                 Building                                                               82.6s 
 ⠦ Service openms-streamlit-template  Building                                                               82.6s  ⠦ Service nuxl-app                   Building                                                               82.6s 
 ⠦ Service FLASHViewer                Building                                                               82.6s 
failed to solve: process "mamba run -n streamlit-env /bin/bash -c git clone --recursive --depth=1 -b ${OPENMS_BRANCH} --single-branch ${OPENMS_REPO} && cd /OpenMS" did not complete successfully: exit code: 128

[t0mdavid-m]

Hey,
thanks for adding StreamSage!

There are currently expected issues with some repositories:

1. Something changed with a miniforge 3 update and the environments have to be activated somewhat differently. This is already fixed in the main repository but needs merging from the relevant repositories.
2. QT was updated from version 5 to 6 in the main repository. This causes builds to fail if they point to develop and the build configuration has not been updated accordingly.

Apps affected by 1:

• NuXL
• Naseweis
• Umetaflow

Apps affected by 2:

• Naseweis

@Arslan-Siraj @poshul @axelwalter

It would be great if you could adress this when you have time! As I discovered the issue just before christmas I applied hotfixes on the deployment server so this is not urgent but should be adressed after the break :)

@singjc I could not recreate the error with git clone so I assume it disappeared or maybe is related to the docker version somehow. I just realized the command in the README.md refers to Compose V1, which is deprecated. I will update it asap.

Here are the commands I used:
docker compose build --no-cache
docker compose up

Your app built successfully (I commented out every repository except yours in the docker-compose.yml due to the issues mentioned above). However, I got the following error when launching the app:

[singjc]

Your app built successfully (I commented out every repository except yours in the docker-compose.yml due to the issues mentioned above). However, I got the following error when launching the app:

Ahh, I forgot to update the StreamSage submodule, it didn't have the StreamSage logo. I just updated it, can you test whether it works now, and can you also test the shipped example data to see if it app works?

[t0mdavid-m]

Looks good to me! Testing the app with example data worked!