From f00112cb48f89dd09ae143bb88c2cfe2d7700a11 Mon Sep 17 00:00:00 2001 From: Anurag Mittal Date: Wed, 15 Jan 2025 16:27:51 +0100 Subject: [PATCH] udpated docs --- README.md | 31 ++++++--------------------- docs/Usage.md | 48 +++++++++++++++++++++++++++++------------- docs/driver-params.md | 49 +++++++++++++++++++++++++++++-------------- 3 files changed, 72 insertions(+), 56 deletions(-) diff --git a/README.md b/README.md index 615fac5..b8eff7e 100644 --- a/README.md +++ b/README.md @@ -3,21 +3,6 @@ The **Scality COSI Driver** integrates Scality RING Object Storage with Kubernetes, leveraging the Kubernetes Container Object Storage Interface (COSI) to enable seamless object storage provisioning and management. This repository provides all necessary resources to deploy, use, and contribute to the Scality COSI Driver. --- - -## Table of Contents - -- [Scality COSI Driver](#scality-cosi-driver) - - [Table of Contents](#table-of-contents) - - [Features](#features) - - [Getting Started](#getting-started) - - [Installation](#installation) - - [Quickstart Guide](#quickstart-guide) - - [Documentation](#documentation) - - [Support](#support) - - [License](#license) - ---- - ## Features | Category | Feature | Notes | @@ -34,20 +19,20 @@ The **Scality COSI Driver** integrates Scality RING Object Storage with Kubernet ### Installation -Follow the [installation guide](docs/installation/install-helm.md) to deploy the Scality COSI Driver using Helm. +Use [Quickstart](#quickstart-guide) or fdollow the [installation guide](docs/installation/install-helm.md) to deploy the Scality COSI Driver using Helm. ### Quickstart Guide To quickly deploy and test the Scality COSI Driver: 1. Ensure your Kubernetes cluster is set up and Helm is installed. -2. Install the COSI CRDs: +2. Create namepsace `container-object-storage-system` and install the COSI controller deployment and COSI CRDs: ```bash kubectl create -k github.com/kubernetes-sigs/container-object-storage-interface ``` -3. Deploy the driver: +3. Deploy the driver: Namespace `container-object-storage-system` will be created in step 2. ```bash helm install scality-cosi-driver oci://ghcr.io/scality/cosi-driver/helm-charts/scality-cosi-driver \ @@ -60,6 +45,8 @@ To quickly deploy and test the Scality COSI Driver: kubectl get pods -n container-object-storage-system ``` +To learn how to use the COSI driver, refer to the [Usage documentation](./docs/usage.md) + --- ## Documentation @@ -69,7 +56,7 @@ The following sections provide detailed documentation for deploying, configuring - **[Installation Guide](docs/installation/install-helm.md):** Step-by-step instructions for deploying the driver. - **[Driver Parameters](docs/driver-params.md):** Configuration options for bucket classes and access credentials. - **[Metrics Overview](docs/metrics-overview.md):** Prometheus metrics exposed by the driver. -- **[Features](docs/features.md):** Detailed guides on bucket provisioning and access control with the COSI driver. +- **[Feature Usage](docs/usage.md):** Detailed guides on bucket provisioning and access control with the COSI driver. - **[Development Documentation](docs/development):** - [Dev Container Setup](docs/development/dev-container-setup.md) - [Remote Debugging](docs/development/remote-debugging-golang-on-kubernetes.md) @@ -80,9 +67,3 @@ The following sections provide detailed documentation for deploying, configuring ## Support For issues, please create a ticket in the [GitHub Issues](https://github.com/scality/cosi-driver/issues) section. - ---- - -## License - -This project is licensed under the [Apache 2.0 License](LICENSE). diff --git a/docs/Usage.md b/docs/Usage.md index 532ce79..e79ef07 100644 --- a/docs/Usage.md +++ b/docs/Usage.md @@ -1,6 +1,6 @@ # Scality COSI Driver Usage Guide: Bucket Provisioning & Access Control -This document provides an overview and step-by-step guidance for implementing **Bucket Provisioning** (both Greenfield and Brownfield) and **Access Control** using the Scality COSI Driver. Example YAML manifests can be found in the [cosi-examples](./cosi-examples/) folder. +This document provides an overview and step-by-step guidance for implementing **Bucket Provisioning** (both Greenfield and Brownfield) and **Access Control** using the Scality COSI Driver. Example YAML manifests can be found in the [cosi-examples](../cosi-examples/) folder. > **Note** > The Scality COSI Driver supports standard AWS S3 and IAM compliant storage solutions like Scality RING, Scality ARTESCA and AWS S3 & IAM @@ -17,6 +17,7 @@ Refer to the quick start guide in the [README](../README.md#quickstart-guide) fo ### Common Setup Steps 1. **Create the IAM User (by the Storage Administrator)** + Create an IAM user and a paid of Access Key ID and Secret Access Key. This user will be used by COSI driver. Assign S3/IAM permissions that allow bucket creation and user management. Permissions needed by COSI driver: - `S3:CreateBucket` - `S3:DeleteBucket` @@ -30,6 +31,7 @@ Refer to the quick start guide in the [README](../README.md#quickstart-guide) fo - `IAM:DeleteAccessKey` 2. **Collect Access & Endpoint Details** + The Storage Administrator provides the below details to the Kubernetes Administrator: - S3 endpoint (and IAM endpoint, if different) @@ -38,6 +40,7 @@ Refer to the quick start guide in the [README](../README.md#quickstart-guide) fo - `tlsCert`, if using HTTPS 3. **Create a Kubernetes Secret (by the Kubernetes Administrator)** + The Kubernetes Administrator creates a secret containing the above credentials and configuration details: ```bash @@ -70,9 +73,13 @@ Refer to the quick start guide in the [README](../README.md#quickstart-guide) fo In the **Scality COSI Driver**, both **Greenfield** and **Brownfield** provisioning share similar steps, with minor differences in how resources (Bucket, BucketClaim) are created. +> Note: +> For **fully working** examples, see the YAMLs in the [cosi-examples/brownfield](./cosi-examples/brownfield/) and [cosi-examples/greenfield](./cosi-examples/greenfield/) directories. +> For brownfield scenario it is madatory to create COSI CRs in the same namespace as COSI driver and controller. + ### 1.1 Greenfield: Creating a New Bucket -Greenfield provisioning will create a brand-new S3 bucket in your object store, managed by Kubernetes. +Greenfield provisioning will create a brand-new S3 bucket in your object store, managed by Kubernetes. Examples can be found [here](../cosi-examples/greenfield/). 1. **Create a BucketClass** A `BucketClass` defines how buckets should be provisioned or deleted. The bucket class name is used as a prefix for bucket name by COSI: @@ -83,6 +90,7 @@ Greenfield provisioning will create a brand-new S3 bucket in your object store, kind: BucketClass metadata: name: greenfield-bucketclass + namespace: container-object-storage-system driverName: cosi.scality.com deletionPolicy: Delete parameters: @@ -91,11 +99,12 @@ Greenfield provisioning will create a brand-new S3 bucket in your object store, EOF ``` - - `driverName` must match the Scality driver (`cosi.scality.com`). + - `driverName` must match the Scality driver (default: `cosi.scality.com`). This determined by driver-prefix in the COSI driver deployment. For more information check [COSI Driver Parameters](./docs/driver-params.md). - `deletionPolicy` can be `Delete` or `Retain`. - - `objectStorageSecretName` and `objectStorageSecretNamespace` reference the secret you created earlier in [Common Setup Steps](#common-setup-steps). + - `objectStorageSecretName` and `objectStorageSecretNamespace` reference the secret you created earlier in [Common Setup Steps](#common-setup-steps). For more information check [COSI Driver Parameters](./docs/driver-params.md). 2. **Create a BucketClaim** + A `BucketClaim` requests a new bucket using the above `BucketClass`: ```bash @@ -104,6 +113,7 @@ Greenfield provisioning will create a brand-new S3 bucket in your object store, kind: BucketClaim metadata: name: my-greenfield-bucketclaim + namespace: container-object-storage-system spec: bucketClassName: greenfield-bucketclass protocols: @@ -111,14 +121,18 @@ Greenfield provisioning will create a brand-new S3 bucket in your object store, EOF ``` - - This will automatically provision a new S3 bucket in the backend. + - This will automatically provision a new S3 bucket in the backend using the COSI driver. The COSI driver will use credentials and endpoint mentioned in the secret specified in the `BucketClass` to create the bucket. - The actual bucket name on S3 is typically generated by the driver (e.g., `-`). + - Only `S3` protocol is supported at the moment. ### 1.2 Brownfield: Using an Existing Bucket Brownfield provisioning allows you to manage an **already-existing** S3 bucket in Kubernetes. +> Note: For brownfield scenario, COSI CRs for Bucket and Access provisioning should be created in the same namespace as COSI driver and controller. + 1. **Verify Existing Bucket** + Ensure the bucket already exists in S3 either through Storage Administrator or by running the following AWS CLI command: ```bash @@ -126,6 +140,7 @@ Brownfield provisioning allows you to manage an **already-existing** S3 bucket i ``` 2. **Create a BucketClass** + Similar to Greenfield, but you will typically still reference the same secret: ```bash @@ -134,6 +149,7 @@ Brownfield provisioning allows you to manage an **already-existing** S3 bucket i kind: BucketClass metadata: name: brownfield-bucketclass + namespace: container-object-storage-system driverName: cosi.scality.com deletionPolicy: Delete parameters: @@ -154,14 +170,19 @@ Brownfield provisioning allows you to manage an **already-existing** S3 bucket i kind: Bucket metadata: name: "" + namespace: container-object-storage-system spec: + bucketClaim: {} driverName: cosi.scality.com bucketClassName: brownfield-bucketclass + driverName: cosi.scality.com deletionPolicy: Retain existingBucketID: "" parameters: objectStorageSecretName: s3-secret-for-cosi objectStorageSecretNamespace: default + protocols: + - S3 EOF ``` @@ -176,7 +197,9 @@ Brownfield provisioning allows you to manage an **already-existing** S3 bucket i kind: BucketClaim metadata: name: my-brownfield-bucketclaim + namespace: container-object-storage-system spec: + bucketClassName: brownfield-bucket-class existingBucketName: "" protocols: - S3 @@ -185,18 +208,10 @@ Brownfield provisioning allows you to manage an **already-existing** S3 bucket i - `existingBucketName` should match the `name` of the `Bucket` object created in the previous step for Bucket Instance. -> For **fully working** examples, see the YAMLs in the [cosi-examples/brownfield](./cosi-examples/brownfield/) and [cosi-examples/greenfield](./cosi-examples/greenfield/) directories. - ### Bucket Provisioning Cleanup To remove the buckets and associated objects: -```bash -kubectl delete bucketclaim my-greenfield-bucketclaim -# OR -kubectl delete bucketclaim my-brownfield-bucketclaim -``` - - **Greenfield**: ```bash @@ -206,13 +221,14 @@ kubectl delete bucketclaim my-brownfield-bucketclaim - Deleting the `BucketClaim` will remove the underlying bucket only if: - `deletionPolicy` was set to `Delete` in `BucketClass`. - The bucket is empty at the time of deletion. + - **Brownfield**: ```bash kubectl delete bucketclaim my-brownfield-bucketclaim ``` - - Deleting the `BucketClaim` and `Bucket` objects in Kubernetes **does not** delete the actual bucket in S3 even if if `deletionPolicy` is `Delete`. + - Deleting the `BucketClaim` and `Bucket` objects in Kubernetes **does not** delete the actual pre-existing bucket in S3 even if if `deletionPolicy` is `Delete`. --- @@ -230,6 +246,7 @@ apiVersion: objectstorage.k8s.io/v1alpha1 kind: BucketAccessClass metadata: name: bucketaccessclass + namespace: container-object-storage-system spec: driverName: cosi.scality.com authenticationType: KEY @@ -254,6 +271,7 @@ apiVersion: objectstorage.k8s.io/v1alpha1 kind: BucketAccess metadata: name: my-bucketaccess + namespace: container-object-storage-system spec: bucketClaimName: my-greenfield-bucketclaim # or my-brownfield-bucketclaim bucketAccessClassName: bucketaccessclass @@ -263,7 +281,7 @@ EOF ``` - `bucketClaimName` references the claim from the previous section. -- `credentialsSecretName` is the name of the secret that will contain the newly generated credentials (Access Key ID / Secret Key). +- `credentialsSecretName` is the name of the secret that will contain the newly generated credentials (Access Key ID / Secret Key). Once the `BucketAccess` is created, the driver will: diff --git a/docs/driver-params.md b/docs/driver-params.md index 0cad7a6..ff6adb0 100644 --- a/docs/driver-params.md +++ b/docs/driver-params.md @@ -13,29 +13,46 @@ The table below details the configuration parameters for BucketClass, which dete ## Configuration Parameters for Kubernetes secret containing S3 credentials and configuration -| **Parameter** | **Description** | **Allowed Values** | **Required** | +| **Parameter** | **Description** | **Allowed Values** | **Required** | |-------------------------------|---------------------------------------------------------------------------------------------------------|---------------------------------------------|--------------| -| `accessKeyId` | The Access Key ID of the identity with S3 bucket creation privileges. | `string` | Yes | -| `secretAccessKey` | The Secret Access Key corresponding to the above Access Key ID. | `string` | Yes | -| `endpoint` | The S3 endpoint URL. If HTTPS is used without a TLS certificate, an insecure connection will be used. | `string` (e.g., `https://s3.ring.internal`) | Yes | -| `region` | The S3 region to use. | `string` (e.g., `us-east-1`) | Yes | -| `tlsCert`| The name of the secret containing the TLS certificate (optional). | `string` | No | -| `iamEndpoint` | The IAM endpoint URL. If not specified endpoint is used as IAMendpoint | `string` (e.g., `https://iam.ring.internal`) | No | - +| `accessKeyId` | The Access Key ID of the identity with S3 bucket creation privileges. | `string` | Yes | +| `secretAccessKey` | The Secret Access Key corresponding to the above Access Key ID. | `string` | Yes | +| `endpoint` | The S3 endpoint URL. If HTTPS is used without a TLS certificate, an insecure connection will be used. | `string` (e.g., `https://s3.ring.internal`) | Yes | +| `region` | The S3 region to use. | `string` (e.g., `us-east-1`) | Yes | +| `tlsCert`| PEM encoded TLS certificate (optional). | `string` | No | +| `iamEndpoint` | The IAM endpoint URL. If not specified endpoint is used as IAMendpoint | `string` (e.g., `https://iam.ring.internal`) | No | [Example](../cosi-examples/s3-secret-for-cosi.yaml) -## Deployment Parameters for COSI Driver +## Deployment Parameters for the Scality COSI Driver Below are the deployment parameters for configuring the COSI driver, which can be passed as flags or environment variables. -| **Parameter** | **Description** | **Default Value** | **Required** | -|---------------------------------|----------------------------------------------------------------|----------------------------------|--------------| -| `driver-address` | The socket file address for the COSI driver. | `unix:///var/lib/cosi/cosi.sock` | Yes | -| `driver-prefix` | The prefix for the COSI driver (e.g., `.scality.com`). | `cosi` | No | -| `driver-metrics-address` | The address to expose Prometheus metrics. | `:8080` | No | -| `driver-metrics-path` | The HTTP path for exposing metrics. | `/metrics` | No | -| `driver-custom-metrics-prefix` | The prefix for metrics collected by the COSI driver. | `scality_cosi_driver` | No | +| **Parameter** | **Description** | **Default Value** | **Required** | +|---------------------------------|-----------------------------------------------------------------------------------------------|--------------------------------------|--------------| +| `driver-address` | The socket file address for the COSI driver. | `unix:///var/lib/cosi/cosi.sock` | Yes | +| `driver-prefix` | The prefix for the COSI driver (e.g., `.scality.com`). | `cosi` | No | +| `driver-metrics-address` | The address (hostname:port) for exposing Prometheus metrics. | `:8080` | No | +| `driver-metrics-path` | The HTTP path for exposing metrics. | `/metrics` | No | +| `driver-custom-metrics-prefix` | The prefix for metrics collected by the COSI driver. | `scality_cosi_driver` | No | +| `driver-otel-endpoint` | The OpenTelemetry (OTEL) endpoint for exporting traces (if `driver-otel-stdout` is false). | `""` (empty string- disable tracing) | No | +| `driver-otel-stdout` | Enable OpenTelemetry trace export to stdout. Disables the OTEL endpoint if set to `true`. | `false` | No | +| `driver-otel-service-name` | The service name reported in OpenTelemetry traces. | `cosi.scality.com` | No | + +For Helm deployments, these parameters can be set in the [values.yaml](../helm/scality-cosi-driver/values.yaml) file or passed as flags during installation. + +## Notes on OpenTelemetry Parameters + +- **`driver-otel-endpoint`**: + Use this to specify an OTEL collector endpoint such as `otel-collector.local:4318`. + If `driver-otel-stdout` is set to `true`, this endpoint is ignored. + +- **`driver-otel-stdout`**: + If set, trace data is printed to stdout in addition to any logging. + This is useful for local debugging but should generally be disabled in production. + +- **`driver-otel-service-name`**: + Defines how the service is labeled in OTEL-based observability platforms (e.g., Jaeger). ### Notes