Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix 1205 #225

Merged
merged 4 commits into from
Dec 5, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/basic-concepts/basic-concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ The meta module within each IOMesh Block Storage component for metadata manageme

**IOMesh CSI Driver**

The self-developed CSI driver that adheres to [the CSI standard](https://github.com/container-storage-interface/spec/blob/master/spec.md) and utilizes RPC (Remote Procedure Call) to manage persistent volumes, delivering reliable and consistent storage for data applications on Kubernetes. Each Kubernetes persistent volume corresponds to an iSCSI LUN in the IOMesh cluster.
The CSI driver that adheres to [the CSI standard](https://github.com/container-storage-interface/spec/blob/master/spec.md) and utilizes RPC (Remote Procedure Call) to manage persistent volumes, delivering reliable and consistent storage for data applications on Kubernetes. Each Kubernetes persistent volume corresponds to an iSCSI LUN in the IOMesh cluster.

**IOMesh Operator**

Expand Down
14 changes: 7 additions & 7 deletions docs/cluster-operations/replace-failed-disk.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ title: Replace Disk
sidebar_label: Replace Disk
---

You can see the health status of physical disks on the IOMesh Dashboard. If any disk is in `Unhealthy`, `Failing`, or `S.M.A.R.T not passed` state, you need to replace it with a new disk as soon as possible.
The IOMesh Dashboard displays the health status of physical disks for easy monitoring. If any disk is indicated as `Unhealthy`, `Failing`, or `S.M.A.R.T not passed`, you should replace it with a new disk as soon as possible.

**Procedure**

Expand All @@ -23,7 +23,7 @@ You can see the health status of physical disks on the IOMesh Dashboard. If any

3. Run the following command multiple times to verify that there are no ongoing migration or recovery tasks in the cluster.

Ensure that the output value is 0. If there is a field that is not 0, you need to wait for it to become 0.
Ensure that the output value is 0. If any field has a non-zero value, you should wait for it to reach 0.

```shell
/opt/iomesh/iomeshctl summary cluster | egrep "recovers|migrates"
Expand All @@ -49,7 +49,7 @@ You can see the health status of physical disks on the IOMesh Dashboard. If any
pending_recovers_bytes: 0
```

4. View the disk that needs to be replaced. Assume the disk `blockdevice-66312cce9037ae891a099ad83f44d7c9` needs to be replaced.
4. View the disk that requires replacement. In the given example, let's assume that the disk `blockdevice-66312cce9037ae891a099ad83f44d7c9` needs to be replaced.
```shell
kubectl --namespace iomesh-system get bd -o wide
```
Expand Down Expand Up @@ -84,9 +84,9 @@ You can see the health status of physical disks on the IOMesh Dashboard. If any
# ...
```

6. Repeat Step 2 and 3 to verify that there are no ongoing migration or recovery tasks for the cluster.
6. Repeat Step 2 and 3 to verify that there are no ongoing migration or recovery tasks in the cluster.

7. Verify that the block device is in `Unclaimed` state.
7. Verify that the block device is in the `Unclaimed` state.
```shell
kubectl get bd blockdevice-66312cce9037ae891a099ad83f44d7c9 -n iomesh-system
```
Expand All @@ -95,7 +95,7 @@ You can see the health status of physical disks on the IOMesh Dashboard. If any
blockdevice-66312cce9037ae891a099ad83f44d7c9 qtest-k8s-1 /dev/sdc 69793218560 Unclaimed Active 44h
```

8. Unplug the disk. Then the disk will enter `Inactive` state.
8. Unplug the disk. Then the disk will enter the `Inactive` state.

Run the following commands simultaneously to remove the block device and its corresponding `blockdeviceclaim`.

Expand All @@ -107,5 +107,5 @@ You can see the health status of physical disks on the IOMesh Dashboard. If any
kubectl delete bdc blockdevice-66312cce9037ae891a099ad83f44d7c9 -n iomesh-system
kubectl delete bd blockdevice-66312cce9037ae891a099ad83f44d7c9 -n iomesh-system
```
9. Plug the new disk. Refer to [Set Up IOMesh](../deploy-iomesh-cluster/setup-iomesh) to mount it to the IOMesh cluster.
9. Plug the new disk. Refer to [Set Up IOMesh](../deploy-iomesh-cluster/setup-iomesh) for mounting steps.

6 changes: 3 additions & 3 deletions docs/cluster-operations/scale-out-cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ title: Scale Out Cluster
sidebar_label: Scale Out Cluster
---

If you have the Enterprise Edition of IOMesh software, you can scale it out online without disrupting the cluster's operation. However, scaling out is not possible with the Community Edition, which limits the number of meta or chunk pods to three. During scaling out, you can choose to add chunk pods, meta pods, or both simultaneously.
If you have the IOMesh Enterprise edition, you can scale out the cluster online without interrupting its operation. However, scaling out is not possible with the Community edition that only allows a maximum of three meta or chunk pods. When scaling out the cluster, you can choose to add chunk pods, meta pods, or both at the same time.

**Prerequisite**

Expand All @@ -18,7 +18,7 @@ Ensure an adequate number of worker nodes in the Kubernetes cluster. Each worker

To increase the capacity of the IOMesh cluster, you can choose to add chunk pods by following these steps:

- Find `chunk` in `iomesh.yaml`, the default configuration file exported during IOMesh installation, and then edit `replicaCount`. The value of `replicaCount` is the total number of chunk pods after scaling out.
- Locate `chunk` in `iomesh.yaml`, the default configuration file exported during IOMesh installation. Then modify the value of `replicaCount`, which represents the total number of chunk pods.

```yaml
chunk:
Expand Down Expand Up @@ -48,7 +48,7 @@ Ensure an adequate number of worker nodes in the Kubernetes cluster. Each worker

An optional step. When deploying IOMesh, three meta pods are created in the IOMesh cluster by default. If the number of IOMesh nodes in the Kubernetes cluster is equal to or greater than five, it's recommended to increase the number of meta pods from three to five. Note that the number of supported meta pods in the IOMesh cluster should be either three or five.

- Find `meta` in `iomesh.yaml`, the default configuration file exported during IOMesh installation, and then edit `replicaCount`.
- Locate `meta` in `iomesh.yaml`, the default configuration file exported during IOMesh installation. Then modify the value of `replicaCount`, which represents the number of meta pods.

```yaml
meta:
Expand Down
29 changes: 14 additions & 15 deletions docs/deploy-iomesh-cluster/install-iomesh.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ title: Install IOMesh
sidebar_label: Install IOMesh
---

IOMesh can be installed on all Kubernetes platforms using various methods. Choose the installation method based on your environment. If the Kubernetes cluster network cannot connect to the public network, you can opt for custom offline installation.
IOMesh can be installed on all Kubernetes platforms using various methods. Choose the installation method based on your environment. If the Kubernetes cluster network cannot connect to the public network, opt for custom offline installation.

- One-click online installation: Use the default settings in the file without customizing parameters.
- One-click online installation: Use the default settings in the file without custom parameters.
- Custom online installation: Supports custom parameters.
- Custom offline installation: Supports custom parameters.

Expand Down Expand Up @@ -40,9 +40,9 @@ IOMesh can be installed on all Kubernetes platforms using various methods. Choos

> _NOTE:_ IOMesh resources left by running the above commands will be saved for troubleshooting if any error occurs during installation. You can run the command `curl -sSL https://iomesh.run/uninstall_iomesh.sh | sh -` to remove all IOMesh resources from the Kubernetes cluster.

> _NOTE:_ After installing IOMesh, the `prepare-csi` Pod will automatically start on all schedulable nodes in the Kubernetes cluster to install and configure `open-iscsi`. If the installation of `open-iscsi` is successful on all nodes, the system will automatically clean up the `prepare-csi` Pod. However, if the installation of `open-iscsi` fails on any node, [manual configuration of open-iscsi](../appendices/setup-worker-node) is required to determine the cause of the installation failure.
> _NOTE:_ After installing IOMesh, the `prepare-csi` pod will automatically start on all schedulable nodes in the Kubernetes cluster to install and configure `open-iscsi`. If the installation of `open-iscsi` is successful on all nodes, the system will automatically clean up all `prepare-csi` pods. However, if the installation of `open-iscsi` fails on any node, [manual configuration of open-iscsi](../appendices/setup-worker-node) is required to determine the cause of the installation failure.

> _NOTE:_ If `open-iscsi` is manually deleted after installing IOMesh, the `prepare-csi` Pod will not automatically start to install `open-iscsi` when reinstalling IOMesh. In this case, [manual configuration of open-iscsi](../appendices/setup-worker-node) is necessary.
> _NOTE:_ If `open-iscsi` is manually deleted after installing IOMesh, the `prepare-csi` pod will not automatically start to install `open-iscsi` when reinstalling IOMesh. In this case, [manual configuration of open-iscsi](../appendices/setup-worker-node) is necessary.

## Custom Online Installation

Expand Down Expand Up @@ -85,8 +85,7 @@ Make sure the CPU architecture of your Kubernetes cluster is Intel x86_64, Hygon
dataCIDR: "" # Fill in the dataCIDR you configured in Prerequisites.
```

- Set `diskDeploymentMode` according to your [disk configurations](../deploy-iomesh-cluster/prerequisites#hardware-requirements). The system defaults to `hybridFlash`.

- Set `diskDeploymentMode` according to your [disk configurations](../deploy-iomesh-cluster/prerequisites#hardware-requirements). The system has a default value of `hybridFlash`. If your disk configuration is all-flash mode, change the value to `allFlash`.
```yaml
diskDeploymentMode: "hybridFlash" # Set the disk deployment mode.
```
Expand All @@ -105,7 +104,7 @@ Make sure the CPU architecture of your Kubernetes cluster is Intel x86_64, Hygon
edition: "" # If left blank, Community Edition will be installed.
```

- An optional step. The number of IOMesh chunk pods is three by default. If you install IOMesh Enterprise Edition, you can deploy more than three chunk pods
- An optional step. The number of IOMesh chunk pods is three by default. If you install IOMesh Enterprise Edition, you can deploy more than three chunk pods.

```yaml
iomesh:
Expand Down Expand Up @@ -135,7 +134,7 @@ Make sure the CPU architecture of your Kubernetes cluster is Intel x86_64, Hygon

- An optional step. Configure the `podDeletePolicy` field to determine whether the system should automatically delete the Pod and rebuild it on another healthy node when the Kubernetes node that hosts the Pod fails. This configuration applies only to the Pod with an IOMesh-created PVC mounted and the access mode set to `ReadWriteOnly`.

If left unspecified, this field is set to `no-delete-pod` by default, indicating that the system won't automatically delete and rebuild the Pod in case of node failure.
If left unspecified, the value of this field will be set to `no-delete-pod` by default, indicating that the system won't automatically delete and rebuild the Pod in case of node failure.
```yaml
csi-driver:
driver:
Expand Down Expand Up @@ -216,9 +215,9 @@ Make sure the CPU architecture of your Kubernetes cluster is Intel x86_64, Hygon
operator-87bb89877-kfs9d 1/1 Running 0 3m23s
operator-87bb89877-z9tfr 1/1 Running 0 3m23s
```
> _NOTE:_ After installing IOMesh, the `prepare-csi` Pod will automatically start on all schedulable nodes in the Kubernetes cluster to install and configure `open-iscsi`. If the installation of `open-iscsi` is successful on all nodes, the system will automatically clean up the `prepare-csi` Pod. However, if the installation of `open-iscsi` fails on any node, [manual configuration of open-iscsi](../appendices/setup-worker-node) is required to determine the cause of the installation failure.
> _NOTE:_ After installing IOMesh, the `prepare-csi` pod will automatically start on all schedulable nodes in the Kubernetes cluster to install and configure `open-iscsi`. If the installation of `open-iscsi` is successful on all nodes, the system will automatically clean up all `prepare-csi` pods. However, if the installation of `open-iscsi` fails on any node, [manual configuration of open-iscsi](../appendices/setup-worker-node) is required to determine the cause of the installation failure.

> _NOTE:_ If `open-iscsi` is manually deleted after installing IOMesh, the `prepare-csi` Pod will not automatically start to install `open-iscsi` when reinstalling IOMesh. In this case, [manual configuration of open-iscsi](../appendices/setup-worker-node) is necessary.
> _NOTE:_ If `open-iscsi` is manually deleted after installing IOMesh, the `prepare-csi` pod will not automatically start to install `open-iscsi` when reinstalling IOMesh. In this case, [manual configuration of open-iscsi](../appendices/setup-worker-node) is necessary.
## Custom Offline Installation

**Prerequisite**
Expand Down Expand Up @@ -273,7 +272,7 @@ Make sure the CPU architecture of your Kubernetes cluster is Intel x86_64, Hygon
dataCIDR: "" # Fill in the dataCIDR you configured previously in Prerequisites.
```

- Set `diskDeploymentMode` according to your [disk configurations](../deploy-iomesh-cluster/prerequisites#hardware-requirements). The system defaults to `hybridFlash`.
- Set `diskDeploymentMode` according to your [disk configurations](../deploy-iomesh-cluster/prerequisites#hardware-requirements). The system has a default value of `hybridFlash`. If your disk configuration is all-flash mode, change the value to `allFlash`.

```yaml
diskDeploymentMode: "hybridFlash" # Set the disk deployment mode.
Expand Down Expand Up @@ -320,9 +319,9 @@ Make sure the CPU architecture of your Kubernetes cluster is Intel x86_64, Hygon
```
It is recommended that you only configure `values`. For more configurations, refer to [Pod Affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity).

- An optional step. Configure the `podDeletePolicy` field to determine whether the system should automatically delete the Pod and rebuild it on another healthy node when the Kubernetes node that hosts the Pod fails. This configuration applies only to the Pod with an IOMesh-created PVC mounted and the access mode set to `ReadWriteOnly`.
- An optional step. Configure the `podDeletePolicy` field to determine whether the system should automatically delete the pod and rebuild it on another healthy node when the Kubernetes node that hosts the pod fails. This configuration applies only to the pod with an IOMesh-created PVC mounted and the access mode set to `ReadWriteOnly`.

If left unspecified, this field is set to `no-delete-pod` by default, indicating that the system won't automatically delete and rebuild the Pod in case of node failure.
If left unspecified, the value of this field will be set to `no-delete-pod` by default, indicating that the system won't automatically delete and rebuild the pod in case of node failure.
```yaml
csi-driver:
driver:
Expand Down Expand Up @@ -396,9 +395,9 @@ Make sure the CPU architecture of your Kubernetes cluster is Intel x86_64, Hygon
operator-85877979-s94vz 1/1 Running 0 2m8s
operator-85877979-xqtml 1/1 Running 0 2m8s
```
> _NOTE:_ After installing IOMesh, the `prepare-csi` Pod will automatically start on all schedulable nodes in the Kubernetes cluster to install and configure `open-iscsi`. If the installation of `open-iscsi` is successful on all nodes, the system will automatically clean up the `prepare-csi` Pod. However, if the installation of `open-iscsi` fails on any node, [manual configuration of open-iscsi](../appendices/setup-worker-node) is required to determine the cause of the installation failure.
> _NOTE:_ After installing IOMesh, the `prepare-csi` pod will automatically start on all schedulable nodes in the Kubernetes cluster to install and configure `open-iscsi`. If the installation of `open-iscsi` is successful on all nodes, the system will automatically clean up all `prepare-csi` Pods. However, if the installation of `open-iscsi` fails on any node, [manual configuration of open-iscsi](../appendices/setup-worker-node) is required to determine the cause of the installation failure.

> _NOTE:_ If `open-iscsi` is manually deleted after installing IOMesh, the `prepare-csi` Pod will not automatically start to install `open-iscsi` when reinstalling IOMesh. In this case, [manual configuration of open-iscsi](../appendices/setup-worker-node) is necessary.
> _NOTE:_ If `open-iscsi` is manually deleted after installing IOMesh, the `prepare-csi` pod will not automatically start to install `open-iscsi` when reinstalling IOMesh. In this case, [manual configuration of open-iscsi](../appendices/setup-worker-node) is necessary.



2 changes: 1 addition & 1 deletion docs/deploy-iomesh-cluster/setup-iomesh.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ Before configuring device map, familiarize yourself with the mount type and devi
|Mode|Mount Type|
|---|---|
|`hybridFlash`|Must configure `cacheWithJournal` and `dataStore`. <ur><li>`cacheWithJournal` serves the performance layer of storage pool and **MUST** be a partitionable block device with a capacity greater than 60 GB. Two partitions will be created: one for journal and the other for cache. Either SATA or NVMe SSD is recommended.</li><li>`dataStore` is used for the capacity layer of storage pool. Either SATA or SAS HDD is recommended.</li></ur>|
|`allflash`|<p>Only need to configure `dataStoreWithJournal`. </p> `dataStoreWithJournal` is used for the capacity layer of storage pool. It **MUST** be a partitionable block device with a capacity greater than 60 GB. Two partitions will be created: one for `journal` and the other for `dataStore`. Either SATA or NVMe SSD is recommended.|
|`allFlash`|<p>Only need to configure `dataStoreWithJournal`. </p> `dataStoreWithJournal` is used for the capacity layer of storage pool. It **MUST** be a partitionable block device with a capacity greater than 60 GB. Two partitions will be created: one for `journal` and the other for `dataStore`. Either SATA or NVMe SSD is recommended.|

**Device Selector**
|Parameter|Value|Description|
Expand Down
Loading