Skip to content

Commit

Permalink
udpated docs
Browse files Browse the repository at this point in the history
  • Loading branch information
@anurag4DSB
anurag4DSB committed Jan 15, 2025
1 parent 76c7b86 commit f00112c
Show file tree
Hide file tree
Showing 3 changed files with 72 additions and 56 deletions.
31 changes: 6 additions & 25 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,6 @@
The **Scality COSI Driver** integrates Scality RING Object Storage with Kubernetes, leveraging the Kubernetes Container Object Storage Interface (COSI) to enable seamless object storage provisioning and management. This repository provides all necessary resources to deploy, use, and contribute to the Scality COSI Driver.

---

## Table of Contents

- [Scality COSI Driver](#scality-cosi-driver)
- [Table of Contents](#table-of-contents)
- [Features](#features)
- [Getting Started](#getting-started)
- [Installation](#installation)
- [Quickstart Guide](#quickstart-guide)
- [Documentation](#documentation)
- [Support](#support)
- [License](#license)

---

## Features

| Category | Feature | Notes |
Expand All @@ -34,20 +19,20 @@ The **Scality COSI Driver** integrates Scality RING Object Storage with Kubernet

### Installation

Follow the [installation guide](docs/installation/install-helm.md) to deploy the Scality COSI Driver using Helm.
Use [Quickstart](#quickstart-guide) or fdollow the [installation guide](docs/installation/install-helm.md) to deploy the Scality COSI Driver using Helm.

### Quickstart Guide

To quickly deploy and test the Scality COSI Driver:

1. Ensure your Kubernetes cluster is set up and Helm is installed.
2. Install the COSI CRDs:
2. Create namepsace `container-object-storage-system` and install the COSI controller deployment and COSI CRDs:

```bash
kubectl create -k github.com/kubernetes-sigs/container-object-storage-interface
```

3. Deploy the driver:
3. Deploy the driver: Namespace `container-object-storage-system` will be created in step 2.

```bash
helm install scality-cosi-driver oci://ghcr.io/scality/cosi-driver/helm-charts/scality-cosi-driver \
Expand All @@ -60,6 +45,8 @@ To quickly deploy and test the Scality COSI Driver:
kubectl get pods -n container-object-storage-system
```

To learn how to use the COSI driver, refer to the [Usage documentation](./docs/usage.md)

---

## Documentation
Expand All @@ -69,7 +56,7 @@ The following sections provide detailed documentation for deploying, configuring
- **[Installation Guide](docs/installation/install-helm.md):** Step-by-step instructions for deploying the driver.
- **[Driver Parameters](docs/driver-params.md):** Configuration options for bucket classes and access credentials.
- **[Metrics Overview](docs/metrics-overview.md):** Prometheus metrics exposed by the driver.
- **[Features](docs/features.md):** Detailed guides on bucket provisioning and access control with the COSI driver.
- **[Feature Usage](docs/usage.md):** Detailed guides on bucket provisioning and access control with the COSI driver.
- **[Development Documentation](docs/development):**
- [Dev Container Setup](docs/development/dev-container-setup.md)
- [Remote Debugging](docs/development/remote-debugging-golang-on-kubernetes.md)
Expand All @@ -80,9 +67,3 @@ The following sections provide detailed documentation for deploying, configuring
## Support

For issues, please create a ticket in the [GitHub Issues](https://github.com/scality/cosi-driver/issues) section.

---

## License

This project is licensed under the [Apache 2.0 License](LICENSE).
48 changes: 33 additions & 15 deletions docs/Usage.md
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Scality COSI Driver Usage Guide: Bucket Provisioning & Access Control

This document provides an overview and step-by-step guidance for implementing **Bucket Provisioning** (both Greenfield and Brownfield) and **Access Control** using the Scality COSI Driver. Example YAML manifests can be found in the [cosi-examples](./cosi-examples/) folder.
This document provides an overview and step-by-step guidance for implementing **Bucket Provisioning** (both Greenfield and Brownfield) and **Access Control** using the Scality COSI Driver. Example YAML manifests can be found in the [cosi-examples](../cosi-examples/) folder.

> **Note**
> The Scality COSI Driver supports standard AWS S3 and IAM compliant storage solutions like Scality RING, Scality ARTESCA and AWS S3 & IAM
Expand All @@ -17,6 +17,7 @@ Refer to the quick start guide in the [README](../README.md#quickstart-guide) fo
### Common Setup Steps

1. **Create the IAM User (by the Storage Administrator)**

Create an IAM user and a paid of Access Key ID and Secret Access Key. This user will be used by COSI driver. Assign S3/IAM permissions that allow bucket creation and user management. Permissions needed by COSI driver:
- `S3:CreateBucket`
- `S3:DeleteBucket`
Expand All @@ -30,6 +31,7 @@ Refer to the quick start guide in the [README](../README.md#quickstart-guide) fo
- `IAM:DeleteAccessKey`

2. **Collect Access & Endpoint Details**

The Storage Administrator provides the below details to the Kubernetes Administrator:

- S3 endpoint (and IAM endpoint, if different)
Expand All @@ -38,6 +40,7 @@ Refer to the quick start guide in the [README](../README.md#quickstart-guide) fo
- `tlsCert`, if using HTTPS

3. **Create a Kubernetes Secret (by the Kubernetes Administrator)**

The Kubernetes Administrator creates a secret containing the above credentials and configuration details:

```bash
Expand Down Expand Up @@ -70,9 +73,13 @@ Refer to the quick start guide in the [README](../README.md#quickstart-guide) fo
In the **Scality COSI Driver**, both **Greenfield** and **Brownfield** provisioning share similar steps, with minor differences in how resources (Bucket, BucketClaim) are created.
> Note:
> For **fully working** examples, see the YAMLs in the [cosi-examples/brownfield](./cosi-examples/brownfield/) and [cosi-examples/greenfield](./cosi-examples/greenfield/) directories.
> For brownfield scenario it is madatory to create COSI CRs in the same namespace as COSI driver and controller.
### 1.1 Greenfield: Creating a New Bucket
Greenfield provisioning will create a brand-new S3 bucket in your object store, managed by Kubernetes.
Greenfield provisioning will create a brand-new S3 bucket in your object store, managed by Kubernetes. Examples can be found [here](../cosi-examples/greenfield/).
1. **Create a BucketClass**
A `BucketClass` defines how buckets should be provisioned or deleted. The bucket class name is used as a prefix for bucket name by COSI:
Expand All @@ -83,6 +90,7 @@ Greenfield provisioning will create a brand-new S3 bucket in your object store,
kind: BucketClass
metadata:
name: greenfield-bucketclass
namespace: container-object-storage-system
driverName: cosi.scality.com
deletionPolicy: Delete
parameters:
Expand All @@ -91,11 +99,12 @@ Greenfield provisioning will create a brand-new S3 bucket in your object store,
EOF
```
- `driverName` must match the Scality driver (`cosi.scality.com`).
- `driverName` must match the Scality driver (default: `cosi.scality.com`). This determined by driver-prefix in the COSI driver deployment. For more information check [COSI Driver Parameters](./docs/driver-params.md).
- `deletionPolicy` can be `Delete` or `Retain`.
- `objectStorageSecretName` and `objectStorageSecretNamespace` reference the secret you created earlier in [Common Setup Steps](#common-setup-steps).
- `objectStorageSecretName` and `objectStorageSecretNamespace` reference the secret you created earlier in [Common Setup Steps](#common-setup-steps). For more information check [COSI Driver Parameters](./docs/driver-params.md).
2. **Create a BucketClaim**
A `BucketClaim` requests a new bucket using the above `BucketClass`:
```bash
Expand All @@ -104,28 +113,34 @@ Greenfield provisioning will create a brand-new S3 bucket in your object store,
kind: BucketClaim
metadata:
name: my-greenfield-bucketclaim
namespace: container-object-storage-system
spec:
bucketClassName: greenfield-bucketclass
protocols:
- S3
EOF
```
- This will automatically provision a new S3 bucket in the backend.
- This will automatically provision a new S3 bucket in the backend using the COSI driver. The COSI driver will use credentials and endpoint mentioned in the secret specified in the `BucketClass` to create the bucket.
- The actual bucket name on S3 is typically generated by the driver (e.g., `<bucketclassName>-<UUID>`).
- Only `S3` protocol is supported at the moment.
### 1.2 Brownfield: Using an Existing Bucket
Brownfield provisioning allows you to manage an **already-existing** S3 bucket in Kubernetes.
> Note: For brownfield scenario, COSI CRs for Bucket and Access provisioning should be created in the same namespace as COSI driver and controller.
1. **Verify Existing Bucket**
Ensure the bucket already exists in S3 either through Storage Administrator or by running the following AWS CLI command:
```bash
aws s3api head-bucket --bucket <EXISTING_BUCKET_NAME> --endpoint-url <S3_ENDPOINT>
```
2. **Create a BucketClass**
Similar to Greenfield, but you will typically still reference the same secret:
```bash
Expand All @@ -134,6 +149,7 @@ Brownfield provisioning allows you to manage an **already-existing** S3 bucket i
kind: BucketClass
metadata:
name: brownfield-bucketclass
namespace: container-object-storage-system
driverName: cosi.scality.com
deletionPolicy: Delete
parameters:
Expand All @@ -154,14 +170,19 @@ Brownfield provisioning allows you to manage an **already-existing** S3 bucket i
kind: Bucket
metadata:
name: "<EXISTING_BUCKET_NAME>"
namespace: container-object-storage-system
spec:
bucketClaim: {}
driverName: cosi.scality.com
bucketClassName: brownfield-bucketclass
driverName: cosi.scality.com
deletionPolicy: Retain
existingBucketID: "<EXISTING_BUCKET_NAME>"
parameters:
objectStorageSecretName: s3-secret-for-cosi
objectStorageSecretNamespace: default
protocols:
- S3
EOF
```
Expand All @@ -176,7 +197,9 @@ Brownfield provisioning allows you to manage an **already-existing** S3 bucket i
kind: BucketClaim
metadata:
name: my-brownfield-bucketclaim
namespace: container-object-storage-system
spec:
bucketClassName: brownfield-bucket-class
existingBucketName: "<EXISTING_BUCKET_NAME>"
protocols:
- S3
Expand All @@ -185,18 +208,10 @@ Brownfield provisioning allows you to manage an **already-existing** S3 bucket i
- `existingBucketName` should match the `name` of the `Bucket` object created in the previous step for Bucket Instance.
> For **fully working** examples, see the YAMLs in the [cosi-examples/brownfield](./cosi-examples/brownfield/) and [cosi-examples/greenfield](./cosi-examples/greenfield/) directories.
### Bucket Provisioning Cleanup
To remove the buckets and associated objects:
```bash
kubectl delete bucketclaim my-greenfield-bucketclaim
# OR
kubectl delete bucketclaim my-brownfield-bucketclaim
```
- **Greenfield**:
```bash
Expand All @@ -206,13 +221,14 @@ kubectl delete bucketclaim my-brownfield-bucketclaim
- Deleting the `BucketClaim` will remove the underlying bucket only if:
- `deletionPolicy` was set to `Delete` in `BucketClass`.
- The bucket is empty at the time of deletion.
- **Brownfield**:
```bash
kubectl delete bucketclaim my-brownfield-bucketclaim
```
- Deleting the `BucketClaim` and `Bucket` objects in Kubernetes **does not** delete the actual bucket in S3 even if if `deletionPolicy` is `Delete`.
- Deleting the `BucketClaim` and `Bucket` objects in Kubernetes **does not** delete the actual pre-existing bucket in S3 even if if `deletionPolicy` is `Delete`.
---
Expand All @@ -230,6 +246,7 @@ apiVersion: objectstorage.k8s.io/v1alpha1
kind: BucketAccessClass
metadata:
name: bucketaccessclass
namespace: container-object-storage-system
spec:
driverName: cosi.scality.com
authenticationType: KEY
Expand All @@ -254,6 +271,7 @@ apiVersion: objectstorage.k8s.io/v1alpha1
kind: BucketAccess
metadata:
name: my-bucketaccess
namespace: container-object-storage-system
spec:
bucketClaimName: my-greenfield-bucketclaim # or my-brownfield-bucketclaim
bucketAccessClassName: bucketaccessclass
Expand All @@ -263,7 +281,7 @@ EOF
```
- `bucketClaimName` references the claim from the previous section.
- `credentialsSecretName` is the name of the secret that will contain the newly generated credentials (Access Key ID / Secret Key).
- `credentialsSecretName` is the name of the secret that will contain the newly generated credentials (Access Key ID / Secret Key).
Once the `BucketAccess` is created, the driver will:
Expand Down
49 changes: 33 additions & 16 deletions docs/driver-params.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,29 +13,46 @@ The table below details the configuration parameters for BucketClass, which dete

## Configuration Parameters for Kubernetes secret containing S3 credentials and configuration

| **Parameter** | **Description** | **Allowed Values** | **Required** |
| **Parameter** | **Description** | **Allowed Values** | **Required** |
|-------------------------------|---------------------------------------------------------------------------------------------------------|---------------------------------------------|--------------|
| `accessKeyId` | The Access Key ID of the identity with S3 bucket creation privileges. | `string` | Yes |
| `secretAccessKey` | The Secret Access Key corresponding to the above Access Key ID. | `string` | Yes |
| `endpoint` | The S3 endpoint URL. If HTTPS is used without a TLS certificate, an insecure connection will be used. | `string` (e.g., `https://s3.ring.internal`) | Yes |
| `region` | The S3 region to use. | `string` (e.g., `us-east-1`) | Yes |
| `tlsCert`| The name of the secret containing the TLS certificate (optional). | `string` | No |
| `iamEndpoint` | The IAM endpoint URL. If not specified endpoint is used as IAMendpoint | `string` (e.g., `https://iam.ring.internal`) | No |

| `accessKeyId` | The Access Key ID of the identity with S3 bucket creation privileges. | `string` | Yes |
| `secretAccessKey` | The Secret Access Key corresponding to the above Access Key ID. | `string` | Yes |
| `endpoint` | The S3 endpoint URL. If HTTPS is used without a TLS certificate, an insecure connection will be used. | `string` (e.g., `https://s3.ring.internal`) | Yes |
| `region` | The S3 region to use. | `string` (e.g., `us-east-1`) | Yes |
| `tlsCert`| PEM encoded TLS certificate (optional). | `string` | No |
| `iamEndpoint` | The IAM endpoint URL. If not specified endpoint is used as IAMendpoint | `string` (e.g., `https://iam.ring.internal`) | No |

[Example](../cosi-examples/s3-secret-for-cosi.yaml)

## Deployment Parameters for COSI Driver
## Deployment Parameters for the Scality COSI Driver

Below are the deployment parameters for configuring the COSI driver, which can be passed as flags or environment variables.

| **Parameter** | **Description** | **Default Value** | **Required** |
|---------------------------------|----------------------------------------------------------------|----------------------------------|--------------|
| `driver-address` | The socket file address for the COSI driver. | `unix:///var/lib/cosi/cosi.sock` | Yes |
| `driver-prefix` | The prefix for the COSI driver (e.g., `<prefix>.scality.com`). | `cosi` | No |
| `driver-metrics-address` | The address to expose Prometheus metrics. | `:8080` | No |
| `driver-metrics-path` | The HTTP path for exposing metrics. | `/metrics` | No |
| `driver-custom-metrics-prefix` | The prefix for metrics collected by the COSI driver. | `scality_cosi_driver` | No |
| **Parameter** | **Description** | **Default Value** | **Required** |
|---------------------------------|-----------------------------------------------------------------------------------------------|--------------------------------------|--------------|
| `driver-address` | The socket file address for the COSI driver. | `unix:///var/lib/cosi/cosi.sock` | Yes |
| `driver-prefix` | The prefix for the COSI driver (e.g., `<prefix>.scality.com`). | `cosi` | No |
| `driver-metrics-address` | The address (hostname:port) for exposing Prometheus metrics. | `:8080` | No |
| `driver-metrics-path` | The HTTP path for exposing metrics. | `/metrics` | No |
| `driver-custom-metrics-prefix` | The prefix for metrics collected by the COSI driver. | `scality_cosi_driver` | No |
| `driver-otel-endpoint` | The OpenTelemetry (OTEL) endpoint for exporting traces (if `driver-otel-stdout` is false). | `""` (empty string- disable tracing) | No |
| `driver-otel-stdout` | Enable OpenTelemetry trace export to stdout. Disables the OTEL endpoint if set to `true`. | `false` | No |
| `driver-otel-service-name` | The service name reported in OpenTelemetry traces. | `cosi.scality.com` | No |

For Helm deployments, these parameters can be set in the [values.yaml](../helm/scality-cosi-driver/values.yaml) file or passed as flags during installation.

## Notes on OpenTelemetry Parameters

- **`driver-otel-endpoint`**:
Use this to specify an OTEL collector endpoint such as `otel-collector.local:4318`.
If `driver-otel-stdout` is set to `true`, this endpoint is ignored.

- **`driver-otel-stdout`**:
If set, trace data is printed to stdout in addition to any logging.
This is useful for local debugging but should generally be disabled in production.

- **`driver-otel-service-name`**:
Defines how the service is labeled in OTEL-based observability platforms (e.g., Jaeger).

### Notes

Expand Down

0 comments on commit f00112c

Please sign in to comment.