This repo is a Helm chart for building and deploying a Large language model (LLM)-enabled chat application. For more information about helm charts see the official Helm Charts Documentation.
The deployment flow, will create an application instance, a model server and a github repository with all the application contents in the given github organization. See the background section for more information.
- You have a Github APP created with sufficient permissions for the organization that the application repository will be created. Detailed instructions for the github application creation can be found here.
- You need to have access to a cluster for each operation with OpenShift 4, like deploying and testing.
- The Namespace that your application will run is already created in your cluster.
- Your cluster should have Openshift Pipelines Operator installed and should be connected to your Github App's webhook. In case your cluster is not configured yet, check out our installation and setup helm charts doing this.
- A
key/valueSecret is already created in the Namespace that you are planning to install your helm release, containing a Github Token with sufficient access to the given Github Organization. You can find the exact permissions here. Your Secret's name and the Key of the github token will be provided as values to the helm chart.
The chatbot helm chart utilizes two main deployments:
- The model service deployment, based on
llama.cppinference server and related to the ai-lab-recipes model server. - The application deployment, a Streamlit based application to interact with the model service. The application is based on the related Chatbot Template.
Apart from the two main deployments, the gitops & OpenShift Pipelines parts are handled by the following:
- The application-gitops-job which takes care of the application github repository creation.
- The tekton Repository which connects our application with the
pipeline-as-code-controllerand we are able to manage all webhooks received from our Github App.
The helm chart can be directly installed from the OpenShift Dev Console. Check here for more information.
To install the Chatbot AI Sample Helm chart using Helm directly, you can run:
helm upgrade --install <release-name> --namespace <helm-release-and-chatbot-application-namespace> .
The .gitignore file in this repository filters files named private-values.yaml. Thus, you can maintain in
your local fork of this repository a value settings file outside of git management. Copy values.yaml in this directory to private-values.yaml and make any necessary edits to private-values.yaml. Then change your helm invocation to the following:
helm upgrade --install <release-name> --namespace <helm-release-and-chatbot-application-namespace> -f ./private-values.yaml .Below is a table of each value used to configure this chart. Note:
- Your helm release's name will be used to as the name of the application github repository.
| Value | Description | Default | Additional Information |
|---|---|---|---|
application.appPort |
The exposed port of the application | 8501 | |
application.appContainer |
The initial image used for the chatbot application interface. | quay.io/redhat-ai-dev/chatbot:latest |
| Value | Description | Default | Additional Information |
|---|---|---|---|
model.modelServicePort |
The exposed port of the model service. | 8001 | |
model.modelServiceContainer |
The image used for the model service. | quay.io/ai-lab/llamacpp_python:latest |
|
initContainer |
The image used for the initContainer of the model service deployment. | quay.io/redhat-ai-dev/granite-7b-lab:latest |
|
model.modelInitCommand |
The model service initContainer command. | ['/usr/bin/install', '/model/model.file', '/shared/'] |
|
model.modelPath |
The path of the model file inside the model service container. | /model/model.file |
| Value | Description | Default | Additional Information |
|---|---|---|---|
gitops.gitSecretName |
The name of the Secret containing the required Github token. | git-secrets |
|
gitops.gitSecretKeyToken |
The name of the Secret's key with the Github token value. | password |
|
gitops.githubOrgName |
[REQUIRED] The Github Organization name that the chatbot application repository will be created into. |
||
gitops.gitSourceRepo |
The Github Repository with the contents of the ai-lab sample chatbot application. It must be either the redhat-ai-dev/ai-lab-samples or its fork. |
redhat-ai-dev/ai-lab-samples |
|
gitops.gitDefaultBranch |
The default branch for the chatbot application Github repository. | main |
|
gitops.quayAccountName |
[REQUIRED] The quay.io account that the application image will be pushed. |