Add security with istio and keycloak

repo.toml

@@ -60,6 +60,7 @@ project_build_order = [
   "kubernetes",
   "partner-validated",
   "container-toolkit",
+  "secure-services-istio-keycloak",
   "review",
 ]
 
@@ -229,3 +230,14 @@ docs_root = "${root}/review"
 name = "Technical Review"
 version = "0.1.0"
 copyright_start = 2023
+
+[repo_docs.projects.secure-services-istio-keycloak]
+docs_root = "${root}/secure-services-istio-keycloak"
+project = "secure-services-istio-keycloak"
+name = "Securing NVIDIA Services with Istio and Keycloak"
+version = "0.1.0"
+copyright_start = 2024
+
+[repo_docs.projects.secure-services-istio-keycloak.builds.linkcheck]
+build_by_default = false
+output_format = "linkcheck"

secure-services-istio-keycloak/configure.md

@@ -0,0 +1,140 @@
+<!--
+  SPDX-FileCopyrightText: Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+  SPDX-License-Identifier: Apache-2.0
+-->
+
+# Configure RBAC
+
+````{only} not publish_bsp
+```{contents}
+:depth: 2
+:backlinks: none
+:local: true
+```
+````
+
+## Inject Istio
+
+1. Label the namespace to enable Istio injection.
+
+   ```console
+   kubectl label namespace <namespace> istio-injection=enabled --overwrite
+   ```
+
+   Replace the `<namespace>` with your target namespace.
+
+2. Delete the existing pods to recreate them with Istio sidecar containers.
+
+   ```console
+   kubectl delete pod $(kubectl get pods -n <namespace> | awk '{print $1}') -n <namespace>
+   ````
+
+## Deploy Manifests
+
+1. The following sample manifest deploys a gateway and ingress virtual service.
+
+    - Update the target namespace for the virtual service resource.
+    - The sample manifest applies to NVIDIA NIM for LLMs. For other NVIDIA microservices, update the `match` and `route` for the microservice endpoints.
+        - For information about the microservice endpoints, refer to the following documents:
+          - [NIM Inference API Inference](https://docs.nvidia.com/nim/large-language-models/latest/api-reference.html)
+          - [NIM Embedding API Reference](https://docs.nvidia.com/nim/nemo-retriever/text-embedding/latest/reference.html)
+          - [NIM ReRanking API Reference](https://docs.nvidia.com/nim/nemo-retriever/text-reranking/latest/reference.html)
+
+   ```{literalinclude} ./manifests/istio-sample-manifest.yaml
+   :language: yaml
+   ```
+
+2. Apply the manifest.
+
+   ```console
+   kubectl apply -f istio-sample-manifest.yaml
+   ````
+
+3. Determine the Istio ingress gateway node port.
+
+   ```console
+   kubectl get svc -n istio-system | grep ingress
+   ```
+
+   *Example Output*
+
+   ```output
+   istio-ingressgateway   LoadBalancer   10.102.8.149     10.28.234.101   15021:32658/TCP,80:30611/TCP,443:31874/TCP,31400:30160/TCP,15443:32430/TCP   22h
+   ```
+
+4. List the worker IP addresses.
+
+   ```console
+   for node in `kubectl get nodes | awk '{print $1}' | grep -v NAME`; do echo $node ' ' | tr -d '\n'; kubectl describe node $node | grep -i 'internalIP:' | awk '{print $2}'; done
+   ```
+
+   *Example Output*
+
+   ```console
+   nim-test-cluster-03-worker-nbhk9-56b4b888dd-8lpqd  10.120.199.16
+   nim-test-cluster-03-worker-nbhk9-56b4b888dd-hnrxr  10.120.199.23
+   ```
+
+5. The following manifest creates request authentication resources.
+
+    - Update the target namespace.
+    - Modify the issuer in the manifest with one of the preceding IP addresses and preceeding ingress Istio gateway node ports, mapped to port 80.
+
+    ```{literalinclude} ./manifests/requestAuthentication.yaml
+    :language: yaml
+    ```
+
+6. Apply the manifest.
+
+   ```console
+   kubectl apply -f requestAuthentication.yaml
+   ```
+
+7. The following manifest creates an authorization policy resource.
+
+    - Update the target namespace.
+    - Update the rules that apply to the target microservices.
+
+   ```{literalinclude} ./manifests/authorizationPolicy.yaml
+   :language: yaml
+   ```
+
+8. Apply the manifest.
+
+   ```console
+   kubectl apply -f authorizationPolicy.yaml
+   ```
+
+9. Create a token for Keycloak authentication.
+   Update the node IP address and ingress gateway node port.
+
+   ```console
+   TOKEN=`curl -X POST -d "client_id=nvidia-nim" -d "username=nim" -d "password=nvidia123" -d "grant_type=password" "http://10.217.19.114:30611/realms/nvidia-nim-llm/protocol/openid-connect/token"| jq .access_token| tr -d '"' `
+   ```
+
+10. Verify access to the microservice from Keycloak through the Istio gateway.
+
+    ```console
+    curl -v -X POST http://10.217.19.114:30611/v1/completions -H "Authorization: Bearer $TOKEN" -H 'accept: application/json' -H 'Content-Type: application/json' -d '{ "model": "llama-2-13b-chat","prompt": "What is Kubernetes?","max_tokens": 16,"temperature": 1, "n": 1, "stream": false, "stop": "string", "frequency_penalty": 0.0 }'
+    ```
+
+    Update the node IP address and ingress gateway port.
+    Update the model name if it is not `llama-2-13b-chat`.
+
+11. Generate some more data so it can be visualized in the next step on the Kiali dashboard.
+
+    ```console
+    for i in $(seq 1 100); do curl -X POST http://10.217.19.114:30611/v1/chat/completions -H 'accept: application/json' -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' -d '{"model": "llama-2-13b-chat","messages": [{"role": "system","content": "You are a helpful assistant."},{"role": "user", "content": "Hello!"}]}'  -s -o /dev/null; done
+    ```
+
+12. Access the Istio Dashboard, specifying your client system IP address.
+
+    ```console
+    istioctl dashboard kiali --address <system-ip>
+    ```
+
+Access in browser with `system-ip` and port `20001`.
+
+## Conclusion
+
+This architecture offers a robust solution for deploying NVIDIA NeMo MicroServices in a secure, scalable, and efficient manner. Integrating advanced service mesh capabilities with OIDC authentication sets a new standard for building sophisticated AI-driven applications.

secure-services-istio-keycloak/implementation.md

@@ -0,0 +1,201 @@
+<!--
+  SPDX-FileCopyrightText: Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+  SPDX-License-Identifier: Apache-2.0
+-->
+
+# Sample Implementation Details
+
+The service mesh uses an Istio-based service mesh for creating a secure, observable, and highly configurable communication layer.
+OIDC is provided by tools like Keycloak, DEX, or other commercial solutions as an OIDC provider. NVIDIA developed and tested this document using the following installations methods.
+
+```{contents}
+:depth: 2
+:backlinks: none
+:local: true
+```
+
+## Prerequisites
+
+- A Kubernetes cluster and the cluster-admin role.
+  Refer to [](platform-support.md) for information about supported operating systems and Kubernetes platforms.
+
+- A Linux VM or WSL on Windows.
+
+## Service Mesh Installation Using Istio
+
+1. Run the following command to download Istio.
+
+   ```console
+   curl https://raw.githubusercontent.com/istio/istio/release-1.23/release/downloadIstioCandidate.sh | sh -
+   ```
+
+   The download creates a directory with name such as ``istio-1.2x.x``.
+
+2. Change directory into the Istio directory.
+
+   ```console
+   cd istio-1.23.2
+   ```
+
+3. Add `istioctl` to `PATH`.
+
+   ```console
+   export PATH=$PWD/bin:$PATH
+   ```
+
+4. Optional: For VMware Tanzu Kubernetes or Red Hat OpenShift, create a namespace and label the namespace to run privileged pods.
+
+   ```console
+   kubectl create ns istio-system
+   ```
+
+   ```console
+   kubectl label --overwrite ns istio-system pod-security.kubernetes.io/warn=privileged  pod-security.kubernetes.io/enforce=privileged
+   ```
+
+5. Install Istio with the demonstration profile.
+
+   ```console
+   istioctl install --set profile=demo -y
+   ```
+
+6. Determine the storage classes on the cluster.
+   Grafana Loki uses persistent storage and you must specify a storage class.
+
+   ```console
+   kubectl get storageclass
+   ```
+
+7. Update the storage class for Loki.
+
+   ```console
+   sed -i '/accessModes:/i\        storageClassName: <storage-class-name> ' samples/addons/loki.yaml
+   ```
+
+8. Run the below command to install the add-ons like Prometheus, Grafana and Jaeger.
+
+    ```console
+    kubectl rollout status deployment/kiali -n istio-system
+    ```
+
+   ```console
+   kubectl rollout status deployment/kiali -n istio-system
+   ```
+
+## Install OIDC Keycloak
+
+1. Label the default namespace to run the privileged pods.
+
+    ```console
+    kubectl label --overwrite ns default pod-security.kubernetes.io/warn=privileged  pod-security.kubernetes.io/enforce=privileged
+    ```
+
+2. Install KeyCloak.
+
+   ```console
+   kubectl create -f https://raw.githubusercontent.com/keycloak/keycloak-quickstarts/latest/kubernetes/keycloak.yaml
+   ```
+
+### Configure KeyCloak
+
+1. Determine the node port of the Keycloak service.
+
+   ```console
+   kubectl get svc
+   ```
+
+   *Example Output*
+
+   ```console
+   NAME       TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)          AGE
+   keycloak   LoadBalancer   10.108.216.157   10.28.234.100   8080:30111/TCP   21h
+   ```
+
+2. List the worker IP addresses and then use any one of them with the service node port.
+
+   ```console
+   for node in `kubectl get nodes | awk '{print $1}' | grep -v NAME`; do echo $node ' ' | tr -d '\n'; kubectl describe node $node | grep -i 'internalIP:' | awk '{print $2}'; done
+   ```
+
+   *Example Output*
+
+   ```console
+   test-cluster-03-worker-nbhk9-56b4b888dd-8lpqd  10.120.199.16
+   test-cluster-03-worker-nbhk9-56b4b888dd-hnrxr  10.120.199.23
+   ```
+
+
+3. Use any of the worker IP addresses to access the Keycloak administration interface.
+
+   ```console
+   10.120.199.15:30111
+   ```
+
+4. After you access the application, you can see the page like the following.
+   Click on **Administration Console**.
+
+   ![](images/keycloak-1.png)
+
+5. Enter the default credentials as `admin` and `admin` and then sign in.
+
+   ![](images/keycloak-2.png)
+
+6. Create a new **Realm**.
+
+   ![](images/keycloak-3.png)
+
+7. Enter the **Realm Name** as `nvidia-nim` and click **Create**.
+
+   ![](images/keycloak-4.png)
+
+8. Click **Clients** on the navigation bar and then click **Create client**.
+
+   ![](images/keycloak-5.png)
+
+9. Provide **Client ID** as `nvidia-nim` and click **Next** with default values for steps 2 and 3.
+
+   ![](images/keycloak-6.png)
+
+10. Navigate to **Realm roles** on the left side pane and click **Create** to create a role.
+
+    ![](images/keycloak-7.png)
+
+11. Create role with name `chat` and save.
+
+    ![](images/keycloak-8.png)
+
+12. Create another role with name `completions` and save.
+
+    ![](images/keycloak-9.png)
+
+13. Navigate to **Users** on left side pane and click **Add user**.
+
+    ![](images/keycloak-10.png)
+
+14. Create a user with name `nim` and click **Create**.
+
+    ![](images/keycloak-11.png)
+
+    Keycloak displays the `nim` User details page.
+
+15. On the `nim` User details page, click **Credentials** and then click **Set password** to create a password.
+
+    ![](images/keycloak-12.png)
+
+16. Enter the password `nvidia`, set the **Temporary** switch to **Off**, and click **Save**.
+
+    ![](images/keycloak-13.png)
+
+    ![](images/keycloak-14.png)
+
+17. Navigate to the **Role Mapping0** tab for `nimuser` and click **Assign Role**.
+
+    ![](images/keycloak-15.png)
+
+18. Enable **chat** and **completion** roles for `nimuser` and click **Assign**.
+
+    ![](images/keycloak-16.png)
+
+## Next Steps
+
+- Refer to [](./configure.md) to create and validate ingress resources.