[Feature Request]: allow volumes and volumeMounts from ConfigMaps for globally or per-namespace needed files

[shalberd]

Feature description

In notebooks / dashboard, I'd like to mount in as a volume a file from a configmap. This might be either via odh notebook controller or odh dashboard. If via Dashboard, the filenames from which configmaps to which volumeMount targetLocation would best be specified centrally in Dashboard Admin Section. A multidimensional array maybe.

https://access.redhat.com/documentation/de-de/openshift_container_platform/4.4/html/builds/builds-configmaps#builds-configmap-creating-from-files_builds-configmaps

Why done centrally and not via the create new workbench dialog?

There are certain volumes and mounts created from configmaps that are universal to all workbenches. Example include Github config entries allowing the use of proxies or Elyra runtime config files (json) for Kubeflow Pipelines. The Git proxy setting is definitely global per cluster. The runtime config files could be potentially different, as with i.e. Airflow, there can be one or more different Airflow runtimes present per cluster in different namespaces, resulting in different entries in fields of the config file. In that case, I could imagine having a configmap in the data science project namespace that contains the needed file:

{
  "display_name": "Airflow TST Instanz",
  "metadata": {
    "tags": [],
    "description": "Airflow Instanz im Zusammenspiel with Namespaces Workbenches TST",
    "display_name": "Airflow TST Instanz",
    "user_namespace": "my-airflow-namespace",
    "git_type": "GITLAB",
    "github_api_endpoint": "https://gitlab.mycompany.com/",
    "api_endpoint": "https://my-airflow-airflow-namespace.apps.tst.mycompany.com/,",
    "github_repo": "airflowstuff",
    "github_branch": "dags",
    "github_repo_token": "",
    "cos_auth_type": "KUBERNETES_SECRET",
    "cos_endpoint": "https://mys3.mycompany.com/",
    "cos_bucket": "my-elyra-bucket",
    "cos_secret": "my-cos-secret",
    "cos_username": "",
    "cos_password": "",
    "runtime_type": "APACHE_AIRFLOW"
  },
  "schema_name": "airflow"
}

On the other hand, the possibility at GUI to have that configmap selection dropdown plus related fields (i.e. all files in data section of configmap, only certain data section entries / files, target mount location in the container) would also be okay.

Current examples that are common, context base directory always under /opt/app-root/src:

1001350000@jupyter-datascience-sven-elyra-test-0 ~]$ pwd
/opt/app-root/src
[1001350000@jupyter-datascience-sven-elyra-test-0 ~]$ grep -rnw . -e "Instanz"
./.local/share/jupyter/metadata/runtimes/airflow_tst_instanz.json:2:  "display_name": "Airflow TST Instanz",
[1001350000@jupyter-datascience-sven-elyra-test-0 ~]$ pwd
/opt/app-root/src
[1001350000@jupyter-datascience-sven-elyra-test-0 ~]$ grep -rnw . -e "proxy"
./.gitconfig:2: 
[http]
        proxy = http://my-proxy.corp.com:8070/

It would help tremendously mounting in the non-secret parts of the config json via a Configmap pre-rolled-out via devops in all data science projects namespaces, having some basic fields already filled out.

Current manually-created situation, need to repeat for each created Notebook / workbench. It is persistent, as the PVC location is mounted to /opt/app-root/src, but having to re-create this for all workbenches is not nice. I could hard-code it into a custom docker build, but that approach is suboptimal, too. I guess the same could be applied to i.e. Jupyter Git plugin configs, like Git plugin proxy settings and so on, that is in a file as well, which would be nice being able to mount it in.

The concept of creating volumes and referencing a configmap and getting in different items (i.e. files) under the configmap data section is explained here:

https://daein.medium.com/how-to-mount-a-specific-file-configured-in-the-configmap-into-the-pod-8b46d958ca1

Describe alternatives you've considered

Manually baking into container workbench docker image builds the files.
This means, though, having to create different container images for different files.
The solution to mount such files via configmap in the workbench namespace is more elegant.

Anything else?

Slack: https://odh-io.slack.com/archives/C0330L52N22/p1701349038086449

@andrewballantyne @harshad16

[andrewballantyne]

Best guess here is you want something done generically to all notebooks ever created on cluster -- then that's a Notebook Controller thing. I'm a little unsure how we approach this, as not everything is created by the Dashboard -- so having a configuration in our admin section won't supply a clean solution, unless we put it in a shape and format for the Notebook Controller to take it on.

@harshad16 I do not see this as something the Dashboard should own. Our next sync is next Thursday, let us talk there about this and figure out what's the best way forward.

[shalberd]

similar request on Slack:
https://odh-io.slack.com/archives/C0330L52N22/p1702074931185009
As to globally or per workbench from configmap with files in data section: why not both ways?
Dashboard workbench creation GUI can show a dropdown of configmaps with files in them. For individual mount-in into notebook at assembleNotebook, for example.
Basically, not just one volume for a PVC, but also additional volumes from configmaps' contents. All files in the configmap or just select ones. If the mount path is at or above /opt/app-root/src, that should work fine.

[harshad16]

Created a JIRA: https://issues.redhat.com/browse/RHOAIENG-3573
Workbench CR , does allow mount of volume from configmap. link.
I guess now it would more of design question, on how the product should provide this ability in good UX way to the user.
For this, to bring attention to UX experts, moving it to JIRA.
will provide updates here, time to time as we receive