The below picture indicates the current state of the Cloud Pak Deployer, which infrastructures are supported to provision or use OpenShift, the storage classes which can be controlled and the Cloud Paks with cartridges and components. 
To install and run the Cloud Pak Deployer, ensure that either podman or docker is available on your system. These are typically available on various Linux distributions such as Red Hat Enterprise Linux (preferred), Fedora, CentOS, Ubuntu, and MacOS. Note that Docker behaves differently on Windows compared to Linux platforms, potentially causing deployment issues.
If you're working on a Windows workstation without access to a Linux server, you can use VirtualBox to create a Linux virtual machine for deployment.
Once the guest operating system is set up, log in as root. VirtualBox supports port forwarding for easy access to the Linux command line using tools like PuTTY.
On Red Hat Enterprise Linux of CentOS, run the following commands:
yum install -y podman git
+yum clean all
+On MacOS, run the following commands:
brew install podman git
+podman machine create
+podman machine init
+On Ubuntu, debian Based :
apt-get -y install podman
+podman machine create
+podman machine init
+Generally, adhere to the instructions provided to install either podman or docker on your Linux system.
If you clone the repository from the command line, you will need to enter a token when you run the git clone command. You can retrieve your token as follows:
Go to a directory where you want to download the Git repo.
git clone --depth=1 https://github.com/IBM/cloud-pak-deployer.git
+First go to the directory where you cloned the GitHub repository, for example ~/cloud-pak-deployer.
cd cloud-pak-deployer
+Then run the following command to build the container image.
./cp-deploy.sh build
+This process will take 5-10 minutes to complete and it will install all the pre-requisites needed to run the automation, including Ansible, Python and required operating system packages. For the installation to work, the system on which the image is built must be connected to the internet.
To download the Cloud Pak Deployer image from the Quay.io registry, you can use the Docker command-line interface (CLI) or Podman.
podman pull quay.io/cloud-pak-deployer/cloud-pak-deployer
+This command pulls the latest version of the Cloud Pak Deployer image from the Quay.io repository. Once downloaded, you can use this image to deploy Cloud Paks
By default, the above command pulls the latest version of the Cloud Pak Deployer image. If you want to specify a particular version or tag, you can append it to the image name. For example:
podman pull quay.io/cloud-pak-deployer/cloud-pak-deployer:<tag_or_version>
+Replace <tag_or_version> with the specific tag or version you want to download.
There are 3 main steps you need to perform to provision an OpenShift cluster with the desired Cloud Pak(s):
To complete the deployment, you will or may need the following. Details will be provided when you need them.
The server on which you run the Cloud Pak Deployer may not have the necessary clients to interact with the cloud infrastructure, OpenShift, or the installed Cloud Pak. You can run commands using the same container image that runs the deployment of OpenShift and the Cloud Paks through the command line: Open a command line
If you want to destroy the provisioned OpenShift cluster, including the installed Cloud Pak(s), you can do this through the Cloud pak Deployer. Steps can be found here: Destroy the assets
On Amazon Web Services (AWS), OpenShift can be set up in various ways, managed by Red Hat (ROSA) or self-managed. The steps below are applicable to the ROSA (Red Hat OpenShift on AWS) installation. More information about ROSA can be found here: https://aws.amazon.com/rosa/
There are 5 main steps to run the deployer for AWS:
A typical setup of the ROSA cluster is pictured below: 
When deploying ROSA, an external host name and domain name are automatically generated by Amazon Web Services and both the API and Ingress servers can be resolved by external clients. At this stage, one cannot configure the domain name to be used.
Deployer reads the configuration from a directory you set in the CONFIG_DIR environment variable. A status directory (STATUS_DIR environment variable) is used to log activities, store temporary files, scripts. If you use a File Vault (default), the secrets are kept in the $STATUS_DIR/vault directory.
You can find OpenShift and Cloud Pak sample configuration (yaml) files here: sample configuration. For ROSA installations, copy one of ocp-aws-rosa-*.yaml files into the $CONFIG_DIR/config directory. If you also want to install a Cloud Pak, copy one of the cp4*.yaml files.
Example:
mkdir -p $HOME/cpd-config/config
+cp sample-configurations/sample-dynamic/config-samples/ocp-aws-rosa-elastic.yaml $HOME/cpd-config/config/
+cp sample-configurations/sample-dynamic/config-samples/cp4d-471.yaml $HOME/cpd-config/config/
+Cloud Pak Deployer uses the status directory to log its activities and also to keep track of its running state. For a given environment you're provisioning or destroying, you should always specify the same status directory to avoid contention between different deploy runs.
export CONFIG_DIR=$HOME/cpd-config
+export STATUS_DIR=$HOME/cpd-status
+CONFIG_DIR: Directory that holds the configuration, it must have a config subdirectory which contains the configuration yaml files.STATUS_DIR: The directory where the Cloud Pak Deployer keeps all status information and logs files.If the deployer configuration is kept on GitHub, follow the instructions in GitHub configuration.
For special configuration with defaults and dynamic variables, refer to Advanced configuration.
Before you can use ROSA on AWS, you have to enable it if this has not been done already. This can be done as follows:
You will need an Access Key ID and Secret Access Key for the deployer to run rosa commands.
If your account uses temporary security credentials for AWS resources, you must use the Access Key ID, Secret Access Key and Session Token associated with your temporary credentials.
For more information about using temporary security credentials, see https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html.
The temporary credentials must be issued for an IAM role that has sufficient permissions to provision the infrastructure and all other components. More information about required permissions for ROSA cluster can be found here: https://docs.openshift.com/rosa/rosa_planning/rosa-sts-aws-prereqs.html#rosa-sts-aws-prereqs.
An example on how to retrieve the temporary credentials for a user-defined role:
printf "\nexport AWS_ACCESS_KEY_ID=%s\nexport AWS_SECRET_ACCESS_KEY=%s\nexport AWS_SESSION_TOKEN=%s\n" $(aws sts assume-role \
+--role-arn arn:aws:iam::678256850452:role/ocp-sts-role \
+--role-session-name OCPInstall \
+--query "Credentials.[AccessKeyId,SecretAccessKey,SessionToken]" \
+--output text)
+This would return something like the below, which you can then paste into the session running the deployer.
export AWS_ACCESS_KEY_ID=ASIxxxxxxAW
+export AWS_SECRET_ACCESS_KEY=jtLxxxxxxxxxxxxxxxGQ
+export AWS_SESSION_TOKEN=IQxxxxxxxxxxxxxbfQ
+You must set the infrastructure.use_sts to True in the openshift configuration if you need to use the temporary security credentials. Cloud Pak Deployer will then run the rosa create cluster command with the appropriate flag.
To run rosa commands to manage the cluster, the deployer requires the ROSA login token.
This scenario is supported. To enable this feature, please ensure that you take the following steps:
{{ env_id }} to match existing clusterCreate "cluster-admin " password token using the following command:
$ ./cp-deploy.sh vault set -vs={{env_id}}-cluster-admin-password=[YOUR PASSWORD]
+Without these changes, sthe cloud player will fail and you will receive the following error message: "Failed to get the cluster-admin password from the vault".
If you want to pull the Cloud Pak images from the entitled registry (i.e. an online install), or if you want to mirror the images to your private registry, you need to download the entitlement key. You can skip this step if you're installing from a private registry and all Cloud Pak images have already been downloaded to the private registry.
Warning
As stated for the API key, you can choose to download the entitlement key to a file. However, when we reference the entitlement key, we mean the 80+ character string that is displayed, not the file.
export AWS_ACCESS_KEY_ID=your_access_key
+export AWS_SECRET_ACCESS_KEY=your_secret_access_key
+export ROSA_LOGIN_TOKEN="your_rosa_login_token"
+export CP_ENTITLEMENT_KEY=your_cp_entitlement_key
+Optional: If your user does not have permanent administrator access but using temporary credentials, you can set the AWS_SESSION_TOKEN to be used for the AWS CLI.
export AWS_SESSION_TOKEN=your_session_token
+AWS_ACCESS_KEY_ID: This is the AWS Access Key you retrieved above, often this is something like AK1A2VLMPQWBJJQGD6GVAWS_SECRET_ACCESS_KEY: The secret associated with your AWS Access Key, also retrieved aboveAWS_SESSION_TOKEN: The session token that will grant temporary elevated permissionsROSA_LOGIN_TOKEN: The offline access token that was retrieved before. This is a very long string (200+ characters). Make sure you enclose the string in single or double quotes as it may hold special charactersCP_ENTITLEMENT_KEY: This is the entitlement key you acquired as per the instructions above, this is a 80+ character stringWarning
If your AWS_SESSION_TOKEN is expires while the deployer is still running, the deployer may end abnormally. In such case, you can just issue new temporary credentials (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN) and restart the deployer. Alternatively, you can update the 3 vault secrets, respectively aws-access-key, aws-secret-access-key and aws-session-token with new values as they are re-retrieved by the deployer on a regular basis.
In some cases, download of the cloudctl and cpd-cli clients from @IBM will fail because GitHub limits the number of API calls from non-authenticated clients. You can remediate this issue by creating a Personal Access Token on github.com and creating a secret in the vault.
./cp-deploy.sh vault set -vs github-ibm-pat=<your PAT>
+Alternatively, you can set the secret by adding -vs github-ibm-pat=<your PAT> to the ./cp-deploy.sh env apply command.
If you only want to validate the configuration, you can run the dpeloyer with the --check-only argument. This will run the first stage to validate variables and vault secrets and then execute the generators.
./cp-deploy.sh env apply --check-only --accept-all-licenses
+To run the container using a local configuration input directory and a data directory where temporary and state is kept, use the example below. If you don't specify the status directory, the deployer will automatically create a temporary directory. Please note that the status directory will also hold secrets if you have configured a flat file vault. If you lose the directory, you will not be able to make changes to the configuration and adjust the deployment. It is best to specify a permanent directory that you can reuse later. If you specify an existing directory the current user must be the owner of the directory. Failing to do so may cause the container to fail with insufficient permissions.
./cp-deploy.sh env apply --accept-all-licenses
+You can also specify extra variables such as env_id to override the names of the objects referenced in the .yaml configuration files as {{ env_id }}-xxxx. For more information about the extra (dynamic) variables, see advanced configuration.
The --accept-all-licenses flag is optional and confirms that you accept all licenses of the installed cartridges and instances. Licenses must be either accepted in the configuration files or at the command line.
When running the command, the container will start as a daemon and the command will tail-follow the logs. You can press Ctrl-C at any time to interrupt the logging but the container will continue to run in the background.
You can return to view the logs as follows:
./cp-deploy.sh env logs
+Deploying the infrastructure, preparing OpenShift and installing the Cloud Pak will take a long time, typically between 1-5 hours,dependent on which Cloud Pak cartridges you configured. For estimated duration of the steps, refer to Timings.
If you need to interrupt the automation, use CTRL-C to stop the logging output and then use:
./cp-deploy.sh env kill
+If the Cloud Pak Deployer fails, for example because certain infrastructure components are temporarily not available, fix the cause if needed and then just re-run it with the same CONFIG_DIR and STATUS_DIR as well extra variables. The provisioning process has been designed to be idempotent and it will not redo actions that have already completed successfully.
Once the process has finished, it will output the URLs by which you can access the deployed Cloud Pak. You can also find this information under the cloud-paks directory in the status directory you specified.
To retrieve the Cloud Pak URL(s):
cat $STATUS_DIR/cloud-paks/*
+This will show the Cloud Pak URLs:
Cloud Pak for Data URL for cluster pluto-01 and project cpd:
+https://cpd-cpd.apps.pluto-01.pmxz.p1.openshiftapps.com
+The admin password can be retrieved from the vault as follows:
List the secrets in the vault:
./cp-deploy.sh vault list
+This will show something similar to the following:
Secret list for group sample:
+- aws-access-key
+- aws-secret-access-key
+- ibm_cp_entitlement_key
+- rosa-login-token
+- pluto-01-cluster-admin-password
+- cp4d_admin_zen_40_pluto_01
+- all-config
+You can then retrieve the Cloud Pak for Data admin password like this:
./cp-deploy.sh vault get --vault-secret cp4d_admin_zen_40_pluto_01
+PLAY [Secrets] *****************************************************************
+included: /cloud-pak-deployer/automation-roles/99-generic/vault/vault-get-secret/tasks/get-secret-file.yml for localhost
+cp4d_admin_zen_40_pluto_01: gelGKrcgaLatBsnAdMEbmLwGr
+You can find examples of a couple of typical changes you may want to do here: Post-run changes.
On Amazon Web Services (AWS), OpenShift can be set up in various ways, self-managed or managed by Red Hat (ROSA). The steps below are applicable to a self-managed OpenShift installation. The IPI (Installer Provisioned Infrastructure) installer will be used. More information about IPI installation can be found here: https://docs.openshift.com/container-platform/4.12/installing/installing_aws/installing-aws-customizations.html.
There are 5 main steps to run the deploye for AWS:
See the deployer in action in this video: https://ibm.box.com/v/cpd-aws-self-managed
A typical setup of the self-managed OpenShift cluster is pictured below: 
Red Hat OpenShift also supports single-node deployments in which control plane and compute are combined into a single node. Obviously, this type of configuration does not cater for any high availability requirements that are usually part of a production installation, but it does offer a more cost-efficient option for development and testing purposes.
Cloud Pak Deployer can deploy a single-node OpenShift with elastic storage and a sample configuration is provided as part of the deployer.
Warning
When deploying the IBM Cloud Paks on single-node OpenShift, there may be intermittent timeouts as pods are starting up. In those cases, just re-run the deployer with the same configuration and check status of the pods.
Deployer reads the configuration from a directory you set in the CONFIG_DIR environment variable. A status directory (STATUS_DIR environment variable) is used to log activities, store temporary files, scripts. If you use a File Vault (default), the secrets are kept in the $STATUS_DIR/vault directory.
You can find OpenShift and Cloud Pak sample configuration (yaml) files here: sample configuration. For self-managed OpenShift installations, copy one of ocp-aws-self-managed-*.yaml files into the $CONFIG_DIR/config directory. If you also want to install a Cloud Pak, copy one of the cp4*.yaml files.
Example:
mkdir -p $HOME/cpd-config/config
+cp sample-configurations/sample-dynamic/config-samples/ocp-aws-self-managed-elastic.yaml $HOME/cpd-config/config/
+cp sample-configurations/sample-dynamic/config-samples/cp4d-471.yaml $HOME/cpd-config/config/
+Cloud Pak Deployer uses the status directory to log its activities and also to keep track of its running state. For a given environment you're provisioning or destroying, you should always specify the same status directory to avoid contention between different deploy runs.
export CONFIG_DIR=$HOME/cpd-config
+export STATUS_DIR=$HOME/cpd-status
+CONFIG_DIR: Directory that holds the configuration, it must have a config subdirectory which contains the configuration yaml files.STATUS_DIR: The directory where the Cloud Pak Deployer keeps all status information and logs files.If the deployer configuration is kept on GitHub, follow the instructions in GitHub configuration.
For special configuration with defaults and dynamic variables, refer to Advanced configuration.
When deploying a self-managed OpenShift on Amazon web Services, a public hosted zone must be created in the same account as your OpenShift cluster. The domain name or subdomain name registered in the Route53 service must be specifed in the openshift configuration of the deployer.
For more information on acquiring or specifying a domain on AWS, you can refer to https://github.com/openshift/installer/blob/master/docs/user/aws/route53.md.
If you can use your permanent security credentials for the AWS account, you will need an Access Key ID and Secret Access Key for the deployer to setup an OpenShift cluster on AWS.
If your account uses temporary security credentials for AWS resources, you must use the Access Key ID, Secret Access Key and Session Token associated with your temporary credentials.
For more information about using temporary security credentials, see https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html.
The temporary credentials must be issued for an IAM role that has sufficient permissions to provision the infrastructure and all other components. More information about required permissions can be found here: https://docs.openshift.com/container-platform/4.10/authentication/managing_cloud_provider_credentials/cco-mode-sts.html#sts-mode-create-aws-resources-ccoctl.
An example on how to retrieve the temporary credentials for a user-defined role:
printf "\nexport AWS_ACCESS_KEY_ID=%s\nexport AWS_SECRET_ACCESS_KEY=%s\nexport AWS_SESSION_TOKEN=%s\n" $(aws sts assume-role \
+--role-arn arn:aws:iam::678256850452:role/ocp-sts-role \
+--role-session-name OCPInstall \
+--query "Credentials.[AccessKeyId,SecretAccessKey,SessionToken]" \
+--output text)
+Thie would return something like the below, which you can then paste into the session running the deployer.
export AWS_ACCESS_KEY_ID=ASIxxxxxxAW
+export AWS_SECRET_ACCESS_KEY=jtLxxxxxxxxxxxxxxxGQ
+export AWS_SESSION_TOKEN=IQxxxxxxxxxxxxxbfQ
+If the openshift configuration has the infrastructure.credentials_mode set to Manual, Cloud Pak Deployer will automatically configure and run the Cloud Credential Operator utility.
If you want to pull the Cloud Pak images from the entitled registry (i.e. an online install), or if you want to mirror the images to your private registry, you need to download the entitlement key. You can skip this step if you're installing from a private registry and all Cloud Pak images have already been downloaded to the private registry.
Warning
As stated for the API key, you can choose to download the entitlement key to a file. However, when we reference the entitlement key, we mean the 80+ character string that is displayed, not the file.
To install OpenShift you need an OpenShift pull secret which holds your entitlement.
/tmp/ocp_pullsecret.jsonTo obtain access to the OpenShift nodes post-installation, you will need to specify the public SSH key of your server; typically this is ~/.ssh/id_rsa.pub, where ~ is the home directory of your user. If you don't have an SSH key-pair yet, you can generate one using the steps documented here: https://cloud.ibm.com/docs/ssh-keys?topic=ssh-keys-generating-and-using-ssh-keys-for-remote-host-authentication#generating-ssh-keys-on-linux. Alternatively, deployer can generate SSH key-pair automatically if credential ocp-ssh-pub-key is not in the vault.
If you want the Cloud Pak images to be pulled from the entitled registry, set the Cloud Pak entitlement key.
export CP_ENTITLEMENT_KEY=your_cp_entitlement_key
+CP_ENTITLEMENT_KEY: This is the entitlement key you acquired as per the instructions above, this is a 80+ character string. You don't need to set this environment variable when you install the Cloud Pak(s) from a private registryexport AWS_ACCESS_KEY_ID=your_access_key
+export AWS_SECRET_ACCESS_KEY=your_secret_access_key
+Optional: If your user does not have permanent administrator access but using temporary credentials, you can set the AWS_SESSION_TOKEN to be used for the AWS CLI.
export AWS_SESSION_TOKEN=your_session_token
+AWS_ACCESS_KEY_ID: This is the AWS Access Key you retrieved above, often this is something like AK1A2VLMPQWBJJQGD6GVAWS_SECRET_ACCESS_KEY: The secret associated with your AWS Access Key, also retrieved aboveAWS_SESSION_TOKEN: The session token that will grant temporary elevated permissionsWarning
If your AWS_SESSION_TOKEN is expires while the deployer is still running, the deployer may end abnormally. In such case, you can just issue new temporary credentials (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN) and restart the deployer. Alternatively, you can update the 3 vault secrets, respectively aws-access-key, aws-secret-access-key and aws-session-token with new values as they are re-retrieved by the deployer on a regular basis.
You need to store the below credentials in the vault so that the deployer has access to them when installing self-managed OpenShift cluster on AWS.
./cp-deploy.sh vault set \
+ --vault-secret ocp-pullsecret \
+ --vault-secret-file /tmp/ocp_pullsecret.json
+If you want to use your SSH key to access nodes in the cluster, set the Vault secret with the public SSH key.
./cp-deploy.sh vault set \
+ --vault-secret ocp-ssh-pub-key \
+ --vault-secret-file ~/.ssh/id_rsa.pub
+In some cases, download of the cloudctl and cpd-cli clients from @IBM will fail because GitHub limits the number of API calls from non-authenticated clients. You can remediate this issue by creating a Personal Access Token on github.com and creating a secret in the vault.
./cp-deploy.sh vault set -vs github-ibm-pat=<your PAT>
+Alternatively, you can set the secret by adding -vs github-ibm-pat=<your PAT> to the ./cp-deploy.sh env apply command.
If you only want to validate the configuration, you can run the dpeloyer with the --check-only argument. This will run the first stage to validate variables and vault secrets and then execute the generators.
./cp-deploy.sh env apply --check-only --accept-all-licenses
+To run the container using a local configuration input directory and a data directory where temporary and state is kept, use the example below. If you don't specify the status directory, the deployer will automatically create a temporary directory. Please note that the status directory will also hold secrets if you have configured a flat file vault. If you lose the directory, you will not be able to make changes to the configuration and adjust the deployment. It is best to specify a permanent directory that you can reuse later. If you specify an existing directory the current user must be the owner of the directory. Failing to do so may cause the container to fail with insufficient permissions.
./cp-deploy.sh env apply --accept-all-licenses
+You can also specify extra variables such as env_id to override the names of the objects referenced in the .yaml configuration files as {{ env_id }}-xxxx. For more information about the extra (dynamic) variables, see advanced configuration.
The --accept-all-licenses flag is optional and confirms that you accept all licenses of the installed cartridges and instances. Licenses must be either accepted in the configuration files or at the command line.
When running the command, the container will start as a daemon and the command will tail-follow the logs. You can press Ctrl-C at any time to interrupt the logging but the container will continue to run in the background.
You can return to view the logs as follows:
./cp-deploy.sh env logs
+Deploying the infrastructure, preparing OpenShift and installing the Cloud Pak will take a long time, typically between 1-5 hours,dependent on which Cloud Pak cartridges you configured. For estimated duration of the steps, refer to Timings.
If you need to interrupt the automation, use CTRL-C to stop the logging output and then use:
./cp-deploy.sh env kill
+If the Cloud Pak Deployer fails, for example because certain infrastructure components are temporarily not available, fix the cause if needed and then just re-run it with the same CONFIG_DIR and STATUS_DIR as well extra variables. The provisioning process has been designed to be idempotent and it will not redo actions that have already completed successfully.
Once the process has finished, it will output the URLs by which you can access the deployed Cloud Pak. You can also find this information under the cloud-paks directory in the status directory you specified.
To retrieve the Cloud Pak URL(s):
cat $STATUS_DIR/cloud-paks/*
+This will show the Cloud Pak URLs:
Cloud Pak for Data URL for cluster pluto-01 and project cpd (domain name specified was example.com):
+https://cpd-cpd.apps.pluto-01.example.com
+The admin password can be retrieved from the vault as follows:
List the secrets in the vault:
./cp-deploy.sh vault list
+This will show something similar to the following:
Secret list for group sample:
+- aws-access-key
+- aws-secret-access-key
+- ocp-pullsecret
+- ocp-ssh-pub-key
+- ibm_cp_entitlement_key
+- pluto-01-cluster-admin-password
+- cp4d_admin_zen_40_pluto_01
+- all-config
+You can then retrieve the Cloud Pak for Data admin password like this:
./cp-deploy.sh vault get --vault-secret cp4d_admin_zen_40_pluto_01
+PLAY [Secrets] *****************************************************************
+included: /cloud-pak-deployer/automation-roles/99-generic/vault/vault-get-secret/tasks/get-secret-file.yml for localhost
+cp4d_admin_zen_40_pluto_01: gelGKrcgaLatBsnAdMEbmLwGr
+You can find examples of a couple of typical changes you may want to do here: Post-run changes:
On Azure, OpenShift can be set up in various ways, managed by Red Hat (ARO) or self-managed. The steps below are applicable to the ARO (Azure Red Hat OpenShift).
There are 5 main steps to run the deployer for Azure:
A typical setup of the ARO cluster is pictured below: 
When deploying ARO, you can configure the domain name by setting the openshift.domain_name attribute. The resulting domain name is managed by Azure, and it must be unique across all ARO instances deployed in Azure. Both the API and Ingress urls are set to be public in the template, so they can be resolved by external clients. If you want to use a custom domain and don't have one yet, you buy one from Azure: https://learn.microsoft.com/en-us/azure/app-service/manage-custom-dns-buy-domain.
Deployer reads the configuration from a directory you set in the CONFIG_DIR environment variable. A status directory (STATUS_DIR environment variable) is used to log activities, store temporary files, scripts. If you use a File Vault (default), the secrets are kept in the $STATUS_DIR/vault directory.
You can find OpenShift and Cloud Pak sample configuration (yaml) files here: sample configuration. For ARO installations, copy one of ocp-azure-aro*.yaml files into the $CONFIG_DIR/config directory. If you also want to install a Cloud Pak, copy one of the cp4*.yaml files.
Example:
mkdir -p $HOME/cpd-config/config
+cp sample-configurations/sample-dynamic/config-samples/ocp-azure-aro.yaml $HOME/cpd-config/config/
+cp sample-configurations/sample-dynamic/config-samples/cp4d-471.yaml $HOME/cpd-config/config/
+Cloud Pak Deployer uses the status directory to log its activities and also to keep track of its running state. For a given environment you're provisioning or destroying, you should always specify the same status directory to avoid contention between different deploy runs.
export CONFIG_DIR=$HOME/cpd-config
+export STATUS_DIR=$HOME/cpd-status
+CONFIG_DIR: Directory that holds the configuration, it must have a config subdirectory which contains the configuration yaml files.STATUS_DIR: The directory where the Cloud Pak Deployer keeps all status information and logs files.If the deployer configuration is kept on GitHub, follow the instructions in GitHub configuration.
For special configuration with defaults and dynamic variables, refer to Advanced configuration.
Install Azure CLI tool, and run the commands in your operating system.
az command. Ideally, one has to have Contributor permissions on the subscription (Azure resources) and Application administrator role assigned in the Azure Active Directory. See details here.export AZURE_RESOURCE_GROUP=pluto-01-rg
+export AZURE_LOCATION=westeurope
+export AZURE_SP=pluto-01-sp
+AZURE_RESOURCE_GROUP: The Azure resource group that will hold all resources belonging to the cluster: VMs, load balancers, virtual networks, subnets, etc.. Typically you will create a resource group for every OpenShift cluster you provision.AZURE_LOCATION: The Azure location of the resource group, for example useast or westeurope.AZURE_SP: Azure service principal that is used to create the resources on Azure. You will get the service principal from the Azure administrator.You must run the OpenShift installation using an Azure Service Principal with sufficient permissions. The Azure account administrator will share the SP credentials as a JSON file. If you have subscription-level access you can also create the Service Principal yourself. See steps in Create Azure service principal.
Example output in credentials file:
{
+ "appId": "a4c39ae9-f9d1-4038-b4a4-ab011e769111",
+ "displayName": "pluto-01-sp",
+ "password": "xyz-xyz",
+ "tenant": "869930ac-17ee-4dda-bbad-7354c3e7629c8"
+}
+Store this file as /tmp/${AZURE_SP}-credentials.json.
Login as the service principal:
az login --service-principal -u a4c39ae9-f9d1-4038-b4a4-ab011e769111 -p xyz-xyz --tenant 869930ac-17ee-4dda-bbad-7354c3e7629c8
+Make sure the following Resource Providers are registered for your subscription by running:
az provider register -n Microsoft.RedHatOpenShift --wait
+az provider register -n Microsoft.Compute --wait
+az provider register -n Microsoft.Storage --wait
+az provider register -n Microsoft.Authorization --wait
+First the resource group must be created; this resource group must match the one configured in your OpenShift yaml config file.
az group create \
+ --name ${AZURE_RESOURCE_GROUP} \
+ --location ${AZURE_LOCATION}
+If you want to pull the Cloud Pak images from the entitled registry (i.e. an online install), or if you want to mirror the images to your private registry, you need to download the entitlement key. You can skip this step if you're installing from a private registry and all Cloud Pak images have already been downloaded to the private registry.
Warning
As stated for the API key, you can choose to download the entitlement key to a file. However, when we reference the entitlement key, we mean the 80+ character string that is displayed, not the file.
To install OpenShift you need an OpenShift pull secret which holds your entitlement.
/tmp/ocp_pullsecret.jsonYou need to store the OpenShift pull secret and service principal credentials in the vault so that the deployer has access to it.
./cp-deploy.sh vault set \
+ --vault-secret ocp-pullsecret \
+ --vault-secret-file /tmp/ocp_pullsecret.json
+
+
+./cp-deploy.sh vault set \
+ --vault-secret ${AZURE_SP}-credentials \
+ --vault-secret-file /tmp/${AZURE_SP}-credentials.json
+In some cases, download of the cloudctl and cpd-cli clients from @IBM will fail because GitHub limits the number of API calls from non-authenticated clients. You can remediate this issue by creating a Personal Access Token on github.com and creating a secret in the vault.
./cp-deploy.sh vault set -vs github-ibm-pat=<your PAT>
+Alternatively, you can set the secret by adding -vs github-ibm-pat=<your PAT> to the ./cp-deploy.sh env apply command.
If you only want to validate the configuration, you can run the dpeloyer with the --check-only argument. This will run the first stage to validate variables and vault secrets and then execute the generators.
./cp-deploy.sh env apply --check-only --accept-all-licenses
+To run the container using a local configuration input directory and a data directory where temporary and state is kept, use the example below. If you don't specify the status directory, the deployer will automatically create a temporary directory. Please note that the status directory will also hold secrets if you have configured a flat file vault. If you lose the directory, you will not be able to make changes to the configuration and adjust the deployment. It is best to specify a permanent directory that you can reuse later. If you specify an existing directory the current user must be the owner of the directory. Failing to do so may cause the container to fail with insufficient permissions.
./cp-deploy.sh env apply --accept-all-licenses
+You can also specify extra variables such as env_id to override the names of the objects referenced in the .yaml configuration files as {{ env_id }}-xxxx. For more information about the extra (dynamic) variables, see advanced configuration.
The --accept-all-licenses flag is optional and confirms that you accept all licenses of the installed cartridges and instances. Licenses must be either accepted in the configuration files or at the command line.
When running the command, the container will start as a daemon and the command will tail-follow the logs. You can press Ctrl-C at any time to interrupt the logging but the container will continue to run in the background.
You can return to view the logs as follows:
./cp-deploy.sh env logs
+Deploying the infrastructure, preparing OpenShift and installing the Cloud Pak will take a long time, typically between 1-5 hours,dependent on which Cloud Pak cartridges you configured. For estimated duration of the steps, refer to Timings.
If you need to interrupt the automation, use CTRL-C to stop the logging output and then use:
./cp-deploy.sh env kill
+If the Cloud Pak Deployer fails, for example because certain infrastructure components are temporarily not available, fix the cause if needed and then just re-run it with the same CONFIG_DIR and STATUS_DIR as well extra variables. The provisioning process has been designed to be idempotent and it will not redo actions that have already completed successfully.
Once the process has finished, it will output the URLs by which you can access the deployed Cloud Pak. You can also find this information under the cloud-paks directory in the status directory you specified.
To retrieve the Cloud Pak URL(s):
cat $STATUS_DIR/cloud-paks/*
+This will show the Cloud Pak URLs:
Cloud Pak for Data URL for cluster pluto-01 and project cpd (domain name specified was example.com):
+https://cpd-cpd.apps.pluto-01.example.com
+The admin password can be retrieved from the vault as follows:
List the secrets in the vault:
./cp-deploy.sh vault list
+This will show something similar to the following:
Secret list for group sample:
+- ibm_cp_entitlement_key
+- sample-provision-ssh-key
+- sample-provision-ssh-pub-key
+- cp4d_admin_zen_sample_sample
+You can then retrieve the Cloud Pak for Data admin password like this:
./cp-deploy.sh vault get --vault-secret cp4d_admin_zen_sample_sample
+PLAY [Secrets] *****************************************************************
+included: /automation_script/automation-roles/99-generic/vault/vault-get-secret/tasks/get-secret-file.yml for localhost
+cp4d_admin_zen_sample_sample: gelGKrcgaLatBsnAdMEbmLwGr
+You can find examples of a couple of typical changes you may want to do here: Post-run changes.
On Azure, OpenShift can be set up in various ways, managed by Red Hat (ARO) or self-managed. The steps below are applicable to the self-managed Red Hat OpenShift.
There are 5 main steps to run the deployer for Azure:
A typical setup of the OpenShift cluster on Azure is pictured below:
When deploying self-managed OpenShift on Azure, you must configure the domain name by setting the openshift.domain_name, which must be public domain with a registrar. OpenShift will create a public DNS zone with additional entries to reach the OpenShift API and the applications (Cloud Paks). If you don't have a domain yet, you buy one from Azure: https://learn.microsoft.com/en-us/azure/app-service/manage-custom-dns-buy-domain.
Deployer reads the configuration from a directory you set in the CONFIG_DIR environment variable. A status directory (STATUS_DIR environment variable) is used to log activities, store temporary files, scripts. If you use a File Vault (default), the secrets are kept in the $STATUS_DIR/vault directory.
You can find OpenShift and Cloud Pak sample configuration (yaml) files here: sample configuration. For Azure self-managed installations, copy one of ocp-azure-self-managed*.yaml files into the $CONFIG_DIR/config directory. If you also want to install a Cloud Pak, copy one of the cp4*.yaml files.
Example:
mkdir -p $HOME/cpd-config/config
+cp sample-configurations/sample-dynamic/config-samples/ocp-azure-self-managed.yaml $HOME/cpd-config/config/
+cp sample-configurations/sample-dynamic/config-samples/cp4d-471.yaml $HOME/cpd-config/config/
+Cloud Pak Deployer uses the status directory to log its activities and also to keep track of its running state. For a given environment you're provisioning or destroying, you should always specify the same status directory to avoid contention between different deploy runs.
export CONFIG_DIR=$HOME/cpd-config
+export STATUS_DIR=$HOME/cpd-status
+CONFIG_DIR: Directory that holds the configuration, it must have a config subdirectory which contains the configuration yaml files.STATUS_DIR: The directory where the Cloud Pak Deployer keeps all status information and logs files.If the deployer configuration is kept on GitHub, follow the instructions in GitHub configuration.
For special configuration with defaults and dynamic variables, refer to Advanced configuration.
Install Azure CLI tool, and run the commands in your operating system.
Contributor permissions on the subscription (Azure resources) and Application administrator role assigned in the Azure Active Directory. See details here.export AZURE_RESOURCE_GROUP=pluto-01-rg
+export AZURE_LOCATION=westeurope
+export AZURE_SP=pluto-01-sp
+AZURE_RESOURCE_GROUP: The Azure resource group that will hold all resources belonging to the cluster: VMs, load balancers, virtual networks, subnets, etc.. Typically you will create a resource group for every OpenShift cluster you provision.AZURE_LOCATION: The Azure location of the resource group, for example useast or westeurope.AZURE_SP: Azure service principal that is used to create the resources on Azure. You will get the service principal from the Azure administrator.You must run the OpenShift installation using an Azure Service Principal with sufficient permissions. The Azure account administrator will share the SP credentials as a JSON file. If you have subscription-level access you can also create the Service Principal yourself. See steps in Create Azure service principal.
Example output in credentials file:
{
+ "appId": "a4c39ae9-f9d1-4038-b4a4-ab011e769111",
+ "displayName": "pluto-01-sp",
+ "password": "xyz-xyz",
+ "tenant": "869930ac-17ee-4dda-bbad-7354c3e7629c8"
+}
+Store this file as /tmp/${AZURE_SP}-credentials.json.
Login as the service principal:
az login --service-principal -u a4c39ae9-f9d1-4038-b4a4-ab011e769111 -p xyz-xyz --tenant 869930ac-17ee-4dda-bbad-7354c3e7629c8
+First the resource group must be created; this resource group must match the one configured in your OpenShift yaml config file.
az group create \
+ --name ${AZURE_RESOURCE_GROUP} \
+ --location ${AZURE_LOCATION}
+If you want to pull the Cloud Pak images from the entitled registry (i.e. an online install), or if you want to mirror the images to your private registry, you need to download the entitlement key. You can skip this step if you're installing from a private registry and all Cloud Pak images have already been downloaded to the private registry.
Warning
As stated for the API key, you can choose to download the entitlement key to a file. However, when we reference the entitlement key, we mean the 80+ character string that is displayed, not the file.
To install OpenShift you need an OpenShift pull secret which holds your entitlement.
/tmp/ocp_pullsecret.jsonTo obtain access to the OpenShift nodes post-installation, you will need to specify the public SSH key of your server; typically this is ~/.ssh/id_rsa.pub, where ~ is the home directory of your user. If you don't have an SSH key-pair yet, you can generate one using the steps documented here: https://cloud.ibm.com/docs/ssh-keys?topic=ssh-keys-generating-and-using-ssh-keys-for-remote-host-authentication#generating-ssh-keys-on-linux. Alternatively, deployer can generate SSH key-pair automatically if credential ocp-ssh-pub-key is not in the vault.
If you want the Cloud Pak images to be pulled from the entitled registry, set the Cloud Pak entitlement key.
export CP_ENTITLEMENT_KEY=your_cp_entitlement_key
+CP_ENTITLEMENT_KEY: This is the entitlement key you acquired as per the instructions above, this is a 80+ character string. You don't need to set this environment variable when you install the Cloud Pak(s) from a private registryYou need to store the OpenShift pull secret and service principal credentials in the vault so that the deployer has access to it.
./cp-deploy.sh vault set \
+ --vault-secret ocp-pullsecret \
+ --vault-secret-file /tmp/ocp_pullsecret.json
+
+
+./cp-deploy.sh vault set \
+ --vault-secret ${AZURE_SP}-credentials \
+ --vault-secret-file /tmp/${AZURE_SP}-credentials.json
+If you want to use your SSH key to access nodes in the cluster, set the Vault secret with the public SSH key.
./cp-deploy.sh vault set \
+ --vault-secret ocp-ssh-pub-key \
+ --vault-secret-file ~/.ssh/id_rsa.pub
+In some cases, download of the cloudctl and cpd-cli clients from @IBM will fail because GitHub limits the number of API calls from non-authenticated clients. You can remediate this issue by creating a Personal Access Token on github.com and creating a secret in the vault.
./cp-deploy.sh vault set -vs github-ibm-pat=<your PAT>
+Alternatively, you can set the secret by adding -vs github-ibm-pat=<your PAT> to the ./cp-deploy.sh env apply command.
If you only want to validate the configuration, you can run the dpeloyer with the --check-only argument. This will run the first stage to validate variables and vault secrets and then execute the generators.
./cp-deploy.sh env apply --check-only --accept-all-licenses
+To run the container using a local configuration input directory and a data directory where temporary and state is kept, use the example below. If you don't specify the status directory, the deployer will automatically create a temporary directory. Please note that the status directory will also hold secrets if you have configured a flat file vault. If you lose the directory, you will not be able to make changes to the configuration and adjust the deployment. It is best to specify a permanent directory that you can reuse later. If you specify an existing directory the current user must be the owner of the directory. Failing to do so may cause the container to fail with insufficient permissions.
./cp-deploy.sh env apply --accept-all-licenses
+You can also specify extra variables such as env_id to override the names of the objects referenced in the .yaml configuration files as {{ env_id }}-xxxx. For more information about the extra (dynamic) variables, see advanced configuration.
The --accept-all-licenses flag is optional and confirms that you accept all licenses of the installed cartridges and instances. Licenses must be either accepted in the configuration files or at the command line.
When running the command, the container will start as a daemon and the command will tail-follow the logs. You can press Ctrl-C at any time to interrupt the logging but the container will continue to run in the background.
You can return to view the logs as follows:
./cp-deploy.sh env logs
+Deploying the infrastructure, preparing OpenShift and installing the Cloud Pak will take a long time, typically between 1-5 hours,dependent on which Cloud Pak cartridges you configured. For estimated duration of the steps, refer to Timings.
If you need to interrupt the automation, use CTRL-C to stop the logging output and then use:
./cp-deploy.sh env kill
+If the Cloud Pak Deployer fails, for example because certain infrastructure components are temporarily not available, fix the cause if needed and then just re-run it with the same CONFIG_DIR and STATUS_DIR as well extra variables. The provisioning process has been designed to be idempotent and it will not redo actions that have already completed successfully.
Once the process has finished, it will output the URLs by which you can access the deployed Cloud Pak. You can also find this information under the cloud-paks directory in the status directory you specified.
To retrieve the Cloud Pak URL(s):
cat $STATUS_DIR/cloud-paks/*
+This will show the Cloud Pak URLs:
Cloud Pak for Data URL for cluster pluto-01 and project cpd (domain name specified was example.com):
+https://cpd-cpd.apps.pluto-01.example.com
+The admin password can be retrieved from the vault as follows:
List the secrets in the vault:
./cp-deploy.sh vault list
+This will show something similar to the following:
Secret list for group sample:
+- ibm_cp_entitlement_key
+- sample-provision-ssh-key
+- sample-provision-ssh-pub-key
+- cp4d_admin_cpd_demo
+You can then retrieve the Cloud Pak for Data admin password like this:
./cp-deploy.sh vault get --vault-secret cp4d_admin_zen_sample_sample
+PLAY [Secrets] *****************************************************************
+included: /automation_script/automation-roles/99-generic/vault/vault-get-secret/tasks/get-secret-file.yml for localhost
+cp4d_admin_zen_sample_sample: gelGKrcgaLatBsnAdMEbmLwGr
+You can find examples of a couple of typical changes you may want to do here: Post-run changes.
Login to the Microsoft Azure using your subscription-level credentials.
az login
+If you have a subscription with multiple tenants, use:
az login --tenant <TENANT_ID>
+Example:
az login --tenant 869930ac-17ee-4dda-bbad-7354c3e7629c8
+To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code AXWFQQ5FJ to authenticate.
+[
+ {
+ "cloudName": "AzureCloud",
+ "homeTenantId": "869930ac-17ee-4dda-bbad-7354c3e7629c8",
+ "id": "72281667-6d54-46cb-8423-792d7bcb1234",
+ "isDefault": true,
+ "managedByTenants": [],
+ "name": "Azure Account",
+ "state": "Enabled",
+ "tenantId": "869930ac-17ee-4dda-bbad-7354c3e7629c8",
+ "user": {
+ "name": "your_user@domain.com",
+ "type": "user"
+ }
+ }
+]
+If you have multiple Azure subscriptions, specify the relevant subscription ID: az account set --subscription <SUBSCRIPTION_ID>
You can list the subscriptions via command:
az account subscription list
+[
+ {
+ "authorizationSource": "RoleBased",
+ "displayName": "IBM xxx",
+ "id": "/subscriptions/dcexxx",
+ "state": "Enabled",
+ "subscriptionId": "dcexxx",
+ "subscriptionPolicies": {
+ "locationPlacementId": "Public_2014-09-01",
+ "quotaId": "EnterpriseAgreement_2014-09-01",
+ "spendingLimit": "Off"
+ }
+ }
+]
+Create the service principal that will do the installation and assign the Contributor role
export AZURE_SUBSCRIPTION_ID=72281667-6d54-46cb-8423-792d7bcb1234
+export AZURE_LOCATION=westeurope
+export AZURE_SP=pluto-01-sp
+AZURE_SUBSCRIPTION_ID: The id of your Azure subscription. Once logged in, you can retrieve this using the az account show command.AZURE_LOCATION: The Azure location of the resource group, for example useast or westeurope.AZURE_SP: Azure service principal that is used to create the resources on Azure.az ad sp create-for-rbac \
+ --role Contributor \
+ --name ${AZURE_SP} \
+ --scopes /subscriptions/${AZURE_SUBSCRIPTION_ID} | tee /tmp/${AZURE_SP}-credentials.json
+Example output:
{
+ "appId": "a4c39ae9-f9d1-4038-b4a4-ab011e769111",
+ "displayName": "pluto-01-sp",
+ "password": "xyz-xyz",
+ "tenant": "869930ac-17ee-4dda-bbad-7354c3e7629c8"
+}
+Finally, set the permissions of the service principal to allow creation of the OpenShift cluster
az role assignment create \
+ --role "User Access Administrator" \
+ --assignee-object-id $(az ad sp list --display-name=${AZURE_SP} --query='[].id' -o tsv)
+See the deployer in action deploying IBM watsonx.ai on an existing OpenShift cluster in this video: https://ibm.box.com/v/cpd-wxai-existing-ocp
Log in as a cluster administrator to be able to run the deployer with the correct permissions.
---
+apiVersion: v1
+kind: Namespace
+metadata:
+ creationTimestamp: null
+ name: cloud-pak-deployer
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+ name: cloud-pak-deployer-sa
+ namespace: cloud-pak-deployer
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+ name: system:openshift:scc:privileged
+ namespace: cloud-pak-deployer
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: ClusterRole
+ name: system:openshift:scc:privileged
+subjects:
+- kind: ServiceAccount
+ name: cloud-pak-deployer-sa
+ namespace: cloud-pak-deployer
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+ name: cloud-pak-deployer-cluster-admin
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: ClusterRole
+ name: cluster-admin
+subjects:
+- kind: ServiceAccount
+ name: cloud-pak-deployer-sa
+ namespace: cloud-pak-deployer
+1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 | |
---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: cloud-pak-deployer-config
+ namespace: cloud-pak-deployer
+data:
+ cpd-config.yaml: |
+ global_config:
+ environment_name: demo
+ cloud_platform: existing-ocp
+ confirm_destroy: False
+
+ openshift:
+ - name: cpd-demo
+ ocp_version: "4.15"
+ cluster_name: cpd-demo
+ domain_name: example.com
+ mcg:
+ install: False
+ storage_type: storage-class
+ storage_class: managed-nfs-storage
+ gpu:
+ install: auto
+ openshift_ai:
+ install: auto
+ channel: auto
+ openshift_storage:
+ - storage_name: auto-storage
+ storage_type: auto
+
+ cp4d:
+ - project: cpd
+ openshift_cluster_name: cpd-demo
+ cp4d_version: 5.0.3
+ db2u_limited_privileges: False
+ use_fs_iam: True
+ accept_licenses: True
+ cartridges:
+ - name: cp-foundation
+ license_service:
+ state: disabled
+ threads_per_core: 2
+
+ - name: lite
+
+ - name: scheduler
+ state: removed
+
+ - name: analyticsengine
+ description: Analytics Engine Powered by Apache Spark
+ size: small
+ state: removed
+
+ - name: bigsql
+ description: Db2 Big SQL
+ state: removed
+
+ - name: ca
+ description: Cognos Analytics
+ size: small
+ instances:
+ - name: ca-instance
+ metastore_ref: ca-metastore
+ state: removed
+
+ - name: dashboard
+ description: Cognos Dashboards
+ state: removed
+
+ - name: datagate
+ description: Db2 Data Gate
+ state: removed
+
+ - name: dataproduct
+ description: Data Product Hub
+ state: removed
+
+ - name: datastage-ent
+ description: DataStage Enterprise
+ state: removed
+
+ - name: datastage-ent-plus
+ description: DataStage Enterprise Plus
+ state: removed
+
+ # The default instance is created automatically with the DataStage installation. If you want to create additional instances
+ # uncomment the section below and specify the various scaling options.
+
+ # instances:
+ # - name: ds-instance
+ # # Optional settings
+ # description: "datastage ds-instance"
+ # size: medium
+ # storage_class: efs-nfs-client
+ # storage_size_gb: 60
+ # # Custom Scale options
+ # scale_px_runtime:
+ # replicas: 2
+ # cpu_request: 500m
+ # cpu_limit: 2
+ # memory_request: 2Gi
+ # memory_limit: 4Gi
+ # scale_px_compute:
+ # replicas: 2
+ # cpu_request: 1
+ # cpu_limit: 3
+ # memory_request: 4Gi
+ # memory_limit: 12Gi
+
+ - name: db2
+ description: Db2 OLTP
+ size: small
+ instances:
+ - name: ca-metastore
+ metadata_size_gb: 20
+ data_size_gb: 20
+ backup_size_gb: 20
+ transactionlog_size_gb: 20
+ state: removed
+
+ - name: db2wh
+ description: Db2 Warehouse
+ state: removed
+
+ - name: dmc
+ description: Db2 Data Management Console
+ state: removed
+ instances:
+ - name: data-management-console
+ description: Data Management Console
+ size: medium
+ storage_size_gb: 50
+
+ - name: dods
+ description: Decision Optimization
+ size: small
+ state: removed
+
+ - name: dp
+ description: Data Privacy
+ size: small
+ state: removed
+
+ - name: dpra
+ description: Data Privacy Risk Assessment
+ state: removed
+
+ - name: dv
+ description: Data Virtualization
+ size: small
+ instances:
+ - name: data-virtualization
+ state: removed
+
+ # Please note that for EDB Postgress, a secret edb-postgres-license-key must be created in the vault
+ # before deploying
+ - name: edb_cp4d
+ description: EDB Postgres
+ state: removed
+ instances:
+ - name: instance1
+ version: "15.4"
+ #type: Standard
+ #members: 1
+ #size_gb: 50
+ #resource_request_cpu: 1
+ #resource_request_memory: 4Gi
+ #resource_limit_cpu: 1
+ #resource_limit_memory: 4Gi
+
+ - name: factsheet
+ description: AI Factsheets
+ size: small
+ state: removed
+
+ - name: hee
+ description: Execution Engine for Apache Hadoop
+ size: small
+ state: removed
+
+ - name: mantaflow
+ description: MANTA Automated Lineage
+ size: small
+ state: removed
+
+ - name: match360
+ description: IBM Match 360
+ size: small
+ wkc_enabled: true
+ state: removed
+
+ - name: openpages
+ description: OpenPages
+ state: removed
+
+ # For Planning Analytics, the case version is needed due to defect in olm utils
+ - name: planning-analytics
+ description: Planning Analytics
+ state: removed
+
+ - name: replication
+ description: Data Replication
+ license: IDRC
+ size: small
+ state: removed
+
+ - name: rstudio
+ description: RStudio Server with R 3.6
+ size: small
+ state: removed
+
+ - name: spss
+ description: SPSS Modeler
+ state: removed
+
+ - name: syntheticdata
+ description: Synthetic Data Generator
+ state: removed
+
+ - name: voice-gateway
+ description: Voice Gateway
+ replicas: 1
+ state: removed
+
+ - name: watson-assistant
+ description: Watson Assistant
+ size: small
+ # noobaa_account_secret: noobaa-admin
+ # noobaa_cert_secret: noobaa-s3-serving-cert
+ state: removed
+ instances:
+ - name: wa-instance
+ description: "Watson Assistant instance"
+
+ - name: watson-discovery
+ description: Watson Discovery
+ # noobaa_account_secret: noobaa-admin
+ # noobaa_cert_secret: noobaa-s3-serving-cert
+ state: removed
+ instances:
+ - name: wd-instance
+ description: "Watson Discovery instance"
+
+ - name: watson-openscale
+ description: Watson OpenScale
+ size: small
+ state: removed
+
+ - name: watson-speech
+ description: Watson Speech (STT and TTS)
+ stt_size: xsmall
+ tts_size: xsmall
+ # noobaa_account_secret: noobaa-admin
+ # noobaa_cert_secret: noobaa-s3-serving-cert
+ state: removed
+
+ # Please note that for watsonx.ai, the following pre-requisites exist:
+ # If you want to use foundation models, you neeed to install the Node Feature Discovery and NVIDIA GPU operators.
+ # You can do so by setting the openshift.gpu.install property to auto
+ # OpenShift AI is a requirement for watsonx.ai. You can install this by setting the openshift.openshift_ai.install property to auto
+ - name: watsonx_ai
+ description: watsonx.ai
+ state: removed
+ installation_options:
+ tuning_disabled: true
+ models:
+ - model_id: allam-1-13b-instruct
+ state: removed
+ - model_id: codellama-codellama-34b-instruct-hf
+ state: removed
+ - model_id: elyza-japanese-llama-2-7b-instruct
+ state: removed
+ - model_id: google-flan-ul2
+ state: removed
+ - model_id: google-flan-t5-xl
+ state: removed
+ - model_id: google-flan-t5-xxl
+ state: removed
+ - model_id: eleutherai-gpt-neox-20b
+ state: removed
+ - model_id: ibm-granite-8b-japanese
+ state: removed
+ - model_id: ibm-granite-13b-chat-v1
+ state: removed
+ - model_id: ibm-granite-13b-chat-v2
+ state: removed
+ - model_id: ibm-granite-13b-instruct-v1
+ state: removed
+ - model_id: ibm-granite-13b-instruct-v2
+ state: removed
+ - model_id: ibm-granite-20b-multilingual
+ state: removed
+ - model_id: core42-jais-13b-chat
+ state: removed
+ - model_id: meta-llama-llama-2-13b-chat
+ state: removed
+ - model_id: meta-llama-llama-3-8b-instruct
+ state: removed
+ - model_id: meta-llama-llama-2-70b-chat
+ state: removed
+ - model_id: mncai-llama-2-13b-dpo-v7
+ state: removed
+ - model_id: ibm-mistralai-merlinite-7b
+ state: removed
+ - model_id: ibm-mpt-7b-instruct2
+ state: removed
+ - model_id: mistralai-mixtral-8x7b-instruct-v01
+ state: removed
+ - model_id: ibm-mistralai-mixtral-8x7b-instruct-v01-q
+ state: removed
+ - model_id: bigscience-mt0-xxl
+ state: removed
+ - model_id: bigcode-starcoder
+ state: removed
+
+ - name: watsonx_data
+ description: watsonx.data
+ state: removed
+
+ - name: watsonx_governance
+ description: watsonx.governance
+ state: removed
+ installation_options:
+ installType: all
+ enableFactsheet: true
+ enableOpenpages: true
+ enableOpenscale: true
+
+ - name: watsonx_orchestrate
+ description: watsonx.orchestrate
+ app_connect:
+ app_connect_project: ibm-app-connect
+ app_connect_case_version: 11.5.0
+ app_connect_channel_version: v11.5
+ state: removed
+
+ - name: wca-ansible
+ description: watsxonx Code Assistant for Red Hat Ansible Lightspeed
+ state: removed
+
+ - name: wca-z
+ description: watsxonx Code Assistant for Z
+ state: removed
+
+ # For the IBM Knowledge Catalog, you can specify 3 editions: wkx, ikc_premium, or ikc_standard
+ # Choose the correct IBM Knowledge Catalog edition below
+ - name: wkc
+ description: IBM Knowledge Catalog
+ size: small
+ state: removed
+ installation_options:
+ enableKnowledgeGraph: False
+ enableDataQuality: False
+
+ - name: ikc_premium
+ description: IBM Knowledge Catalog - Premium edition
+ size: small
+ state: removed
+ installation_options:
+ enableKnowledgeGraph: False
+ enableDataQuality: False
+
+ - name: ikc_standard
+ description: IBM Knowledge Catalog - Standard edition
+ size: small
+ state: removed
+ installation_options:
+ enableKnowledgeGraph: False
+ enableDataQuality: False
+
+ - name: wml
+ description: Watson Machine Learning
+ size: small
+ state: installed
+
+ - name: wml-accelerator
+ description: Watson Machine Learning Accelerator
+ replicas: 1
+ size: small
+ state: removed
+
+ - name: ws
+ description: Watson Studio
+ state: installed
+
+ - name: ws-pipelines
+ description: Watson Studio Pipelines
+ state: removed
+
+ - name: ws-runtimes
+ description: Watson Studio Runtimes
+ runtimes:
+ - ibm-cpd-ws-runtime-241-py
+ - ibm-cpd-ws-runtime-231-py
+ - ibm-cpd-ws-runtime-241-pygpu
+ - ibm-cpd-ws-runtime-231-pygpu
+ - ibm-cpd-ws-runtime-241-r
+ - ibm-cpd-ws-runtime-231-r
+ state: removed
+1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14 +15 +16 +17 +18 +19 +20 +21 | |
cloud-pak-deployer as the project at the top of the pageInfo
When running the deployer installing Cloud Pak for Data, the first run will fail. This is because the deployer applies the node configuration to OpenShift, which will cause all nodes to restart one by one, including the node that runs the deployer. Because of the Job setting, a new deployer pod will automatically start and resume from where it was stopped.
If the deployer has failed or if you want to make changes to the configuration after the successful run, you can do the following:
cloud-pak-deployer jobcloud-pak-deployer-config Config Map by going to Workloads → ConfigMapsWhen running the Cloud Pak Deployer on an existing OpenShift cluster, the following is assumed:
--dry-run)Info
If you don't want to make changes to the OpenShift cluster and only want to review the steps deployer will run, you can use the --dry-run option with cp-deploy.sh. This will generate a log file $STATUS_DIR/log/deployer-activities.log, which lists the steps deployer will execute when running without --dry-run. Please note that the dry-run option has only been implemented for Cloud Pak for Data i.e. watsonx.
Info
You can also choose to run Cloud Pak Deployer as a job on the OpenShift cluster. This removes the dependency on a separate server or workstation to run the deployer. Please note that you may need unrestricted OpenShift entitlements for this. To run the deployer on OpenShift via the OpenShift console, see Run on OpenShift using console
With the Existing OpenShift type of deployment you can install and configure the Cloud Pak(s) both on connected and disconnected (air-gapped) cluster. When using the deployer for a disconnected cluster, make sure you specify --air-gapped for the cp-deploy.sh command.
There are 5 main steps to run the deployer for existing OpenShift:
Deployer reads the configuration from a directory you set in the CONFIG_DIR environment variable. A status directory (STATUS_DIR environment variable) is used to log activities, store temporary files, scripts. If you use a File Vault (default), the secrets are kept in the $STATUS_DIR/vault directory.
You can find OpenShift and Cloud Pak sample configuration (yaml) files here: sample configuration. For existing OpenShift installations, copy one of ocp-existing-ocp-*.yaml files into the $CONFIG_DIR/config directory. If you also want to install a Cloud Pak, copy one of the cp4*.yaml files.
Example:
mkdir -p $HOME/cpd-config/config
+cp sample-configurations/sample-dynamic/config-samples/ocp-existing-ocp-auto.yaml $HOME/cpd-config/config/
+cp sample-configurations/sample-dynamic/config-samples/cp4d-471.yaml $HOME/cpd-config/config/
+Cloud Pak Deployer uses the status directory to log its activities and also to keep track of its running state. For a given environment you're provisioning or destroying, you should always specify the same status directory to avoid contention between different deploy runs.
export CONFIG_DIR=$HOME/cpd-config
+export STATUS_DIR=$HOME/cpd-status
+CONFIG_DIR: Directory that holds the configuration, it must have a config subdirectory which contains the configuration yaml files.STATUS_DIR: The directory where the Cloud Pak Deployer keeps all status information and logs files.If the deployer configuration is kept on GitHub, follow the instructions in GitHub configuration.
For special configuration with defaults and dynamic variables, refer to Advanced configuration.
No steps should be required to prepare the infrastructure; this type of installation expects the OpenShift cluster to be up and running with the supported storage classes.
If you want to pull the Cloud Pak images from the entitled registry (i.e. an online install), or if you want to mirror the images to your private registry, you need to download the entitlement key. You can skip this step if you're installing from a private registry and all Cloud Pak images have already been downloaded to the private registry.
Warning
As stated for the API key, you can choose to download the entitlement key to a file. However, when we reference the entitlement key, we mean the 80+ character string that is displayed, not the file.
If you want the Cloud Pak images to be pulled from the entitled registry, set the Cloud Pak entitlement key.
export CP_ENTITLEMENT_KEY=your_cp_entitlement_key
+CP_ENTITLEMENT_KEY: This is the entitlement key you acquired as per the instructions above, this is a 80+ character string. You don't need to set this environment variable when you install the Cloud Pak(s) from a private registryBecause you will be deploying the Cloud Pak on an existing OpenShift cluster, the deployer needs to be able to access OpenShift. There are thre methods for passing the login credentials of your OpenShift cluster(s) to the deployer process:
Regardless of which authentication option you choose, the deployer will retrieve the secret from the vault when it requires access to OpenShift. If the secret cannot be found or if it is invalid or the OpenShift login token has expired, the deployer will fail and you will need to update the secret of your choice.
For most OpenShift installations, you can retrieve the oc login command with a temporary token from the OpenShift console. Go to the OpenShift console and click on your user at the top right of the page to get the login command. Typically this command looks something like this: oc login --server=https://api.pluto-01.coc.ibm.com:6443 --token=sha256~NQUUMroU4B6q_GTBAMS18Y3EIba1KHnJ08L2rBHvTHA
Before passing the oc login command or the kubeconfig file, make sure you can login to your cluster using the command or the config file. If the cluster's API server has a self-signed certificate, make sure you specify the --insecure-skip-tls-verify flag for the oc login command.
Example:
oc login api.pluto-01.coc.ibm.com:6443 -u kubeadmin -p BmxQ5-KjBFx-FgztG-gpTF3 --insecure-skip-tls-verify
+Output:
Login successful.
+
+You have access to 65 projects, the list has been suppressed. You can list all projects with 'oc projects'
+
+Using project "default".
+oc login command🔗This is the most straightforward option if you only have 1 OpenShift cluster in your configuration.
Set the environment variable for the oc login command
export CPD_OC_LOGIN="oc login api.pluto-01.coc.ibm.com:6443 -u kubeadmin -p BmxQ5-KjBFx-FgztG-gpTF3 --insecure-skip-tls-verify"
+Info
Make sure you put the oc login command between quotes (single or double) to make sure the full command is stored.
When the deployer is run, it automatically sets the oc-login vault secret to the specified oc login command. When logging in to OpenShift, the deployer first checks if there is a specific oc login secret for the cluster in question (see option 2). If there is not, it will default to the generic oc-login secret (option 1).
oc login command(s)🔗Use this option if you have multiple OpenShift clusters configured in th deployer configuration.
Store the login command in secret <cluster name>-oc-login
./cp-deploy.sh vault set \
+ -vs pluto-01-oc-login \
+ -vsv "oc login api.pluto-01.coc.ibm.com:6443 -u kubeadmin -p BmxQ5-KjBFx-FgztG-gpTF3 --insecure-skip-tls-verify"
+Info
Make sure you put the oc login command between quotes (single or double) to make sure the full command is stored.
If you already have a "kubeconfig" file that holds the credentials of your cluster, you can use this, otherwise: - Log in to OpenShift as a cluster administrator using your method of choice - Locate the Kubernetes config file. If you have logged in with the OpenShift client, this is typically ~/.kube/config
If you did not just login to the cluster, the current context of the kubeconfig file may not point to your cluster. The deployer will check that the server the current context points to matches the cluster_name and domain_name of the configured openshift object. To check the current context, run the following command:
oc config current-context
+Now, store the Kubernetes config file as a vault secret.
./cp-deploy.sh vault set \
+ --vault-secret kubeconfig \
+ --vault-secret-file ~/.kube/config
+If the deployer manages multiple OpenShift clusters, you can specify a kubeconfig file for each of the clusters by prefixing the kubeconfig with the name of the openshift object, for example:
./cp-deploy.sh vault set \
+ --vault-secret pluto-01-kubeconfig \
+ --vault-secret-file /data/pluto-01/kubeconfig
+
+./cp-deploy.sh vault set \
+ --vault-secret venus-02-kubeconfig \
+ --vault-secret-file /data/venus-02/kubeconfig
+kubeconfig secret. In some cases, download of the cloudctl and cpd-cli clients from @IBM will fail because GitHub limits the number of API calls from non-authenticated clients. You can remediate this issue by creating a Personal Access Token on github.com and creating a secret in the vault.
./cp-deploy.sh vault set -vs github-ibm-pat=<your PAT>
+Alternatively, you can set the secret by adding -vs github-ibm-pat=<your PAT> to the ./cp-deploy.sh env apply command.
If you only want to validate the configuration, you can run the dpeloyer with the --check-only argument. This will run the first stage to validate variables and vault secrets and then execute the generators.
./cp-deploy.sh env apply --check-only --accept-all-licenses
+To run the container using a local configuration input directory and a data directory where temporary and state is kept, use the example below. If you don't specify the status directory, the deployer will automatically create a temporary directory. Please note that the status directory will also hold secrets if you have configured a flat file vault. If you lose the directory, you will not be able to make changes to the configuration and adjust the deployment. It is best to specify a permanent directory that you can reuse later. If you specify an existing directory the current user must be the owner of the directory. Failing to do so may cause the container to fail with insufficient permissions.
./cp-deploy.sh env apply --accept-all-licenses
+You can also specify extra variables such as env_id to override the names of the objects referenced in the .yaml configuration files as {{ env_id }}-xxxx. For more information about the extra (dynamic) variables, see advanced configuration.
The --accept-all-licenses flag is optional and confirms that you accept all licenses of the installed cartridges and instances. Licenses must be either accepted in the configuration files or at the command line.
When running the command, the container will start as a daemon and the command will tail-follow the logs. You can press Ctrl-C at any time to interrupt the logging but the container will continue to run in the background.
You can return to view the logs as follows:
./cp-deploy.sh env logs
+Deploying the infrastructure, preparing OpenShift and installing the Cloud Pak will take a long time, typically between 1-5 hours,dependent on which Cloud Pak cartridges you configured. For estimated duration of the steps, refer to Timings.
If you need to interrupt the automation, use CTRL-C to stop the logging output and then use:
./cp-deploy.sh env kill
+If the Cloud Pak Deployer fails, for example because certain infrastructure components are temporarily not available, fix the cause if needed and then just re-run it with the same CONFIG_DIR and STATUS_DIR as well extra variables. The provisioning process has been designed to be idempotent and it will not redo actions that have already completed successfully.
Once the process has finished, it will output the URLs by which you can access the deployed Cloud Pak. You can also find this information under the cloud-paks directory in the status directory you specified.
To retrieve the Cloud Pak URL(s):
cat $STATUS_DIR/cloud-paks/*
+This will show the Cloud Pak URLs:
Cloud Pak for Data URL for cluster pluto-01 and project cpd (domain name specified was example.com):
+https://cpd-cpd.apps.pluto-01.example.com
+The admin password can be retrieved from the vault as follows:
List the secrets in the vault:
./cp-deploy.sh vault list
+This will show something similar to the following:
Secret list for group sample:
+- ibm_cp_entitlement_key
+- oc-login
+- cp4d_admin_cpd_demo
+You can then retrieve the Cloud Pak for Data admin password like this:
./cp-deploy.sh vault get --vault-secret cp4d_admin_cpd_sample
+PLAY [Secrets] *****************************************************************
+included: /cloud-pak-deployer/automation-roles/99-generic/vault/vault-get-secret/tasks/get-secret-file.yml for localhost
+cp4d_admin_zen_sample_sample: gelGKrcgaLatBsnAdMEbmLwGr
+You can find examples of a couple of typical changes you may want to do here: Post-run changes.
This guide provides detailed instructions on using Cloud Pak Deployer (CPD) on Fusion HCI using Spectrum Scale as the storage backend.
Fusion HCI (Hyper-Converged Infrastructure) is an IBM offering that combines compute, storage, and networking resources into a single, pre-configured system. It simplifies IT infrastructure management and is optimized for deploying containerized applications. Notably, Fusion HCI integrates OpenShift, a leading container orchestration platform, for streamlined development and deployment. The stack is very simple :
Info
This guide detail the process of installing CPD on a Spectrum HCI using Spectrum Scale as the underlying storage solution. Fusion HCI offers the option to deploy ODF but it's not covedered here, follow classic OCP+ODF guides. IBM Storage Fusion HCI System use ibm-storage-fusion-cp-sc storage class and is created by default on this environment.
There are 5 main steps to run the deployer for Fusion HCI
Deployer reads the configuration from a directory you set in the CONFIG_DIR environment variable. A status directory (STATUS_DIR environment variable) is used to log activities, store temporary files, scripts. If you use a File Vault (default), the secrets are kept in the $STATUS_DIR/vault directory.
Discover sample configuration YAML files for OpenShift and Cloud Pak here: sample configuration. To set up FusionHCI, transfer the fusion-hci.yaml files to the $CONFIG_DIR/config directory. If you're interested in installing WatsonX as well, choose one of the watsonx-*.yaml files and copy it accordingly.
Example:
mkdir -p $HOME/cpd-config/config
+cp sample-configurations/sample-dynamic/config-samples/fusion-hci.yaml $HOME/cpd-config/config/
+cp sample-configurations/sample-dynamic/config-samples/watsonx-480.yaml $HOME/cpd-config/config/
+Cloud Pak Deployer uses the status directory to log its activities and also to keep track of its running state. For a given environment you're provisioning or destroying, you should always specify the same status directory to avoid contention between different deploy runs.
export CONFIG_DIR=$HOME/cpd-config
+export STATUS_DIR=$HOME/cpd-status
+CONFIG_DIR: Directory that holds the configuration, it must have a config subdirectory which contains the configuration yaml files.STATUS_DIR: The directory where the Cloud Pak Deployer keeps all status information and logs files.If the deployer configuration is kept on GitHub, follow the instructions in GitHub configuration.
For special configuration with defaults and dynamic variables, refer to Advanced configuration.
No steps should be required to prepare the infrastructure; this type of installation expects the OpenShift cluster to be up and running with the supported storage classes.
If you want to pull the Cloud Pak images from the entitled registry (i.e. an online install), or if you want to mirror the images to your private registry, you need to download the entitlement key. You can skip this step if you're installing from a private registry and all Cloud Pak images have already been downloaded to the private registry.
Warning
As stated for the API key, you can choose to download the entitlement key to a file. However, when we reference the entitlement key, we mean the 80+ character string that is displayed, not the file.
If you want the Cloud Pak images to be pulled from the entitled registry, set the Cloud Pak entitlement key.
export CP_ENTITLEMENT_KEY=your_cp_entitlement_key
+CP_ENTITLEMENT_KEY: This is the entitlement key you acquired as per the instructions above, this is a 80+ character string. You don't need to set this environment variable when you install the Cloud Pak(s) from a private registryBecause you will be deploying the Cloud Pak on an existing OpenShift cluster, the deployer needs to be able to access OpenShift. There are thre methods for passing the login credentials of your OpenShift cluster(s) to the deployer process:
Regardless of which authentication option you choose, the deployer will retrieve the secret from the vault when it requires access to OpenShift. If the secret cannot be found or if it is invalid or the OpenShift login token has expired, the deployer will fail and you will need to update the secret of your choice.
For most OpenShift installations, you can retrieve the oc login command with a temporary token from the OpenShift console. Go to the OpenShift console and click on your user at the top right of the page to get the login command. Typically this command looks something like this: oc login --server=https://api.pluto-01.coc.ibm.com:6443 --token=sha256~NQUUMroU4B6q_GTBAMS18Y3EIba1KHnJ08L2rBHvTHA
Before passing the oc login command or the kubeconfig file, make sure you can login to your cluster using the command or the config file. If the cluster's API server has a self-signed certificate, make sure you specify the --insecure-skip-tls-verify flag for the oc login command.
Example:
oc login api.pluto-01.coc.ibm.com:6443 -u kubeadmin -p BmxQ5-KjBFx-FgztG-gpTF3 --insecure-skip-tls-verify
+Output:
Login successful.
+
+You have access to 65 projects, the list has been suppressed. You can list all projects with 'oc projects'
+
+Using project "default".
+oc login command🔗This is the most straightforward option if you only have 1 OpenShift cluster in your configuration.
Set the environment variable for the oc login command
export CPD_OC_LOGIN="oc login api.pluto-01.coc.ibm.com:6443 -u kubeadmin -p BmxQ5-KjBFx-FgztG-gpTF3 --insecure-skip-tls-verify"
+Info
Make sure you put the oc login command between quotes (single or double) to make sure the full command is stored.
When the deployer is run, it automatically sets the oc-login vault secret to the specified oc login command. When logging in to OpenShift, the deployer first checks if there is a specific oc login secret for the cluster in question (see option 2). If there is not, it will default to the generic oc-login secret (option 1).
oc login command(s)🔗Use this option if you have multiple OpenShift clusters configured in th deployer configuration.
Store the login command in secret <cluster name>-oc-login
./cp-deploy.sh vault set \
+ -vs pluto-01-oc-login \
+ -vsv "oc login api.pluto-01.coc.ibm.com:6443 -u kubeadmin -p BmxQ5-KjBFx-FgztG-gpTF3 --insecure-skip-tls-verify"
+Info
Make sure you put the oc login command between quotes (single or double) to make sure the full command is stored.
If you already have a "kubeconfig" file that holds the credentials of your cluster, you can use this, otherwise: - Log in to OpenShift as a cluster administrator using your method of choice - Locate the Kubernetes config file. If you have logged in with the OpenShift client, this is typically ~/.kube/config
If you did not just login to the cluster, the current context of the kubeconfig file may not point to your cluster. The deployer will check that the server the current context points to matches the cluster_name and domain_name of the configured openshift object. To check the current context, run the following command:
oc config current-context
+Now, store the Kubernetes config file as a vault secret.
./cp-deploy.sh vault set \
+ --vault-secret kubeconfig \
+ --vault-secret-file ~/.kube/config
+If the deployer manages multiple OpenShift clusters, you can specify a kubeconfig file for each of the clusters by prefixing the kubeconfig with the name of the openshift object, for example:
./cp-deploy.sh vault set \
+ --vault-secret pluto-01-kubeconfig \
+ --vault-secret-file /data/pluto-01/kubeconfig
+
+./cp-deploy.sh vault set \
+ --vault-secret venus-02-kubeconfig \
+ --vault-secret-file /data/venus-02/kubeconfig
+kubeconfig secret. If you only want to validate the configuration, you can run the dpeloyer with the --check-only argument. This will run the first stage to validate variables and vault secrets and then execute the generators.
./cp-deploy.sh env apply --check-only --accept-all-licenses
+To run the container using a local configuration input directory and a data directory where temporary and state is kept, use the example below. If you don't specify the status directory, the deployer will automatically create a temporary directory. Please note that the status directory will also hold secrets if you have configured a flat file vault. If you lose the directory, you will not be able to make changes to the configuration and adjust the deployment. It is best to specify a permanent directory that you can reuse later. If you specify an existing directory the current user must be the owner of the directory. Failing to do so may cause the container to fail with insufficient permissions.
./cp-deploy.sh env apply --accept-all-licenses
+You can also specify extra variables such as env_id to override the names of the objects referenced in the .yaml configuration files as {{ env_id }}-xxxx. For more information about the extra (dynamic) variables, see advanced configuration.
The --accept-all-licenses flag is optional and confirms that you accept all licenses of the installed cartridges and instances. Licenses must be either accepted in the configuration files or at the command line.
When running the command, the container will start as a daemon and the command will tail-follow the logs. You can press Ctrl-C at any time to interrupt the logging but the container will continue to run in the background.
You can return to view the logs as follows:
./cp-deploy.sh env logs
+Deploying the infrastructure, preparing OpenShift and installing the Cloud Pak will take a long time, typically between 1-5 hours,dependent on which Cloud Pak cartridges you configured. For estimated duration of the steps, refer to Timings.
If you need to interrupt the automation, use CTRL-C to stop the logging output and then use:
./cp-deploy.sh env kill
+If the Cloud Pak Deployer fails, for example because certain infrastructure components are temporarily not available, fix the cause if needed and then just re-run it with the same CONFIG_DIR and STATUS_DIR as well extra variables. The provisioning process has been designed to be idempotent and it will not redo actions that have already completed successfully.
Once the process has finished, it will output the URLs by which you can access the deployed Cloud Pak. You can also find this information under the cloud-paks directory in the status directory you specified.
To retrieve the Cloud Pak URL(s):
cat $STATUS_DIR/cloud-paks/*
+This will show the Cloud Pak URLs:
Cloud Pak for Data URL for cluster pluto-01 and project cpd (domain name specified was example.com):
+https://cpd-cpd.apps.pluto-01.example.com
+The admin password can be retrieved from the vault as follows:
List the secrets in the vault:
./cp-deploy.sh vault list
+This will show something similar to the following:
Secret list for group sample:
+- ibm_cp_entitlement_key
+- oc-login
+- cp4d_admin_cpd_demo
+You can then retrieve the Cloud Pak for Data admin password like this:
./cp-deploy.sh vault get --vault-secret cp4d_admin_cpd_sample
+PLAY [Secrets] *****************************************************************
+included: /cloud-pak-deployer/automation-roles/99-generic/vault/vault-get-secret/tasks/get-secret-file.yml for localhost
+cp4d_admin_zen_sample_sample: gelGKrcgaLatBsnAdMEbmLwGr
+You can find examples of a couple of typical changes you may want to do here: Post-run changes.
You can use Cloud Pak Deployer to create a ROKS (Red Hat OpenShift Kubernetes Service) on IBM Cloud.
There are 5 main steps to run the deployer for IBM Cloud:
See the deployer in action in this video: https://ibm.box.com/v/cpd-ibm-cloud-roks
A typical setup of the ROKS cluster on IBM Cloud VPC is pictured below: 
Deployer reads the configuration from a directory you set in the CONFIG_DIR environment variable. A status directory (STATUS_DIR environment variable) is used to log activities, store temporary files, scripts. If you use a File Vault (default), the secrets are kept in the $STATUS_DIR/vault directory.
You can find OpenShift and Cloud Pak sample configuration (yaml) files here: sample configuration. For IBM Cloud installations, copy one of ocp-ibm-cloud-roks*.yaml files into the $CONFIG_DIR/config directory. If you also want to install a Cloud Pak, copy one of the cp4*.yaml files.
Example:
mkdir -p $HOME/cpd-config/config
+cp sample-configurations/sample-dynamic/config-samples/ocp-ibm-cloud-roks-ocs.yaml $HOME/cpd-config/config/
+cp sample-configurations/sample-dynamic/config-samples/cp4d-471.yaml $HOME/cpd-config/config/
+Cloud Pak Deployer uses the status directory to log its activities and also to keep track of its running state. For a given environment you're provisioning or destroying, you should always specify the same status directory to avoid contention between different deploy runs.
export CONFIG_DIR=$HOME/cpd-config
+export STATUS_DIR=$HOME/cpd-status
+CONFIG_DIR: Directory that holds the configuration, it must have a config subdirectory which contains the configuration yaml files.STATUS_DIR: The directory where the Cloud Pak Deployer keeps all status information and logs files.If the deployer configuration is kept on GitHub, follow the instructions in GitHub configuration.
For special configuration with defaults and dynamic variables, refer to Advanced configuration.
In order for the Cloud Pak Deployer to create the infrastructure and deploy IBM Cloud Pak for Data, it must perform tasks on IBM Cloud. In order to do so it requires an IBM Cloud API Key. This can be created by following these steps:
Warning
You can choose to download the API key for later reference. However, when we reference the API key, we mean the IBM Cloud API key as a 40+ character string.
Set the environment variables specific to IBM Cloud deployments.
export IBM_CLOUD_API_KEY=your_api_key
+IBM_CLOUD_API_KEY: This is the API key you generated using your IBM Cloud account, this is a 40+ character stringIf you want to pull the Cloud Pak images from the entitled registry (i.e. an online install), or if you want to mirror the images to your private registry, you need to download the entitlement key. You can skip this step if you're installing from a private registry and all Cloud Pak images have already been downloaded to the private registry.
Warning
As stated for the API key, you can choose to download the entitlement key to a file. However, when we reference the entitlement key, we mean the 80+ character string that is displayed, not the file.
If you want the Cloud Pak images to be pulled from the entitled registry, set the Cloud Pak entitlement key.
export CP_ENTITLEMENT_KEY=your_cp_entitlement_key
+CP_ENTITLEMENT_KEY: This is the entitlement key you acquired as per the instructions above, this is a 80+ character string. You don't need to set this environment variable when you install the Cloud Pak(s) from a private registryIn some cases, download of the cloudctl and cpd-cli clients from @IBM will fail because GitHub limits the number of API calls from non-authenticated clients. You can remediate this issue by creating a Personal Access Token on github.com and creating a secret in the vault.
./cp-deploy.sh vault set -vs github-ibm-pat=<your PAT>
+Alternatively, you can set the secret by adding -vs github-ibm-pat=<your PAT> to the ./cp-deploy.sh env apply command.
If you only want to validate the configuration, you can run the dpeloyer with the --check-only argument. This will run the first stage to validate variables and vault secrets and then execute the generators.
./cp-deploy.sh env apply --check-only --accept-all-licenses
+To run the container using a local configuration input directory and a data directory where temporary and state is kept, use the example below. If you don't specify the status directory, the deployer will automatically create a temporary directory. Please note that the status directory will also hold secrets if you have configured a flat file vault. If you lose the directory, you will not be able to make changes to the configuration and adjust the deployment. It is best to specify a permanent directory that you can reuse later. If you specify an existing directory the current user must be the owner of the directory. Failing to do so may cause the container to fail with insufficient permissions.
./cp-deploy.sh env apply --accept-all-licenses
+You can also specify extra variables such as env_id to override the names of the objects referenced in the .yaml configuration files as {{ env_id }}-xxxx. For more information about the extra (dynamic) variables, see advanced configuration.
The --accept-all-licenses flag is optional and confirms that you accept all licenses of the installed cartridges and instances. Licenses must be either accepted in the configuration files or at the command line.
When running the command, the container will start as a daemon and the command will tail-follow the logs. You can press Ctrl-C at any time to interrupt the logging but the container will continue to run in the background.
You can return to view the logs as follows:
./cp-deploy.sh env logs
+Deploying the infrastructure, preparing OpenShift and installing the Cloud Pak will take a long time, typically between 1-5 hours,dependent on which Cloud Pak cartridges you configured. For estimated duration of the steps, refer to Timings.
If you need to interrupt the automation, use CTRL-C to stop the logging output and then use:
./cp-deploy.sh env kill
+If the Cloud Pak Deployer fails, for example because certain infrastructure components are temporarily not available, fix the cause if needed and then just re-run it with the same CONFIG_DIR and STATUS_DIR as well extra variables. The provisioning process has been designed to be idempotent and it will not redo actions that have already completed successfully.
Once the process has finished, it will output the URLs by which you can access the deployed Cloud Pak. You can also find this information under the cloud-paks directory in the status directory you specified.
To retrieve the Cloud Pak URL(s):
cat $STATUS_DIR/cloud-paks/*
+This will show the Cloud Pak URLs:
Cloud Pak for Data URL for cluster pluto-01 and project cpd (domain name specified was example.com):
+https://cpd-cpd.apps.pluto-01.example.com
+The admin password can be retrieved from the vault as follows:
List the secrets in the vault:
./cp-deploy.sh vault list
+This will show something similar to the following:
Secret list for group sample:
+- ibm_cp_entitlement_key
+- sample-provision-ssh-key
+- sample-provision-ssh-pub-key
+- sample-terraform-tfstate
+- cp4d_admin_cpd_demo
+You can then retrieve the Cloud Pak for Data admin password like this:
./cp-deploy.sh vault get --vault-secret cp4d_admin_cpd_demo
+PLAY [Secrets] *****************************************************************
+included: /cloud-pak-deployer/automation-roles/99-generic/vault/vault-get-secret/tasks/get-secret-file.yml for localhost
+cp4d_admin_zen_sample_sample: gelGKrcgaLatBsnAdMEbmLwGr
+You can find examples of a couple of typical changes you may want to do here: Post-run changes.
Cloud Pak Deployer supports various public and private cloud infrastructures. Click on the links below, or in the left menu to find details about running the deployer on each of the following infrastructures:
You can use Cloud Pak Deployer to create an OpenShift cluster on VMWare infrastructure.
There are 5 main steps to run the deployer for vSphere:
A typical setup of the vSphere cluster with OpenShift is pictured below: 
When deploying OpenShift and the Cloud Pak(s) on VMWare vSphere, there is a dependency on a DHCP server for issuing IP addresses to the newly configured cluster nodes. Also, once the OpenShift cluster has been installed, valid fully qualified host names are required to connect to the OpenShift API server at port 6443 and applications running behind the ingress server at port 443. The Cloud Pak deployer cannot set up a DHCP server or a DNS server and to be able to connect to OpenShift or to reach the Cloud Pak after installation, name entries must be set up.
Deployer reads the configuration from a directory you set in the CONFIG_DIR environment variable. A status directory (STATUS_DIR environment variable) is used to log activities, store temporary files, scripts. If you use a File Vault (default), the secrets are kept in the $STATUS_DIR/vault directory.
You can find OpenShift and Cloud Pak sample configuration (yaml) files here: sample configuration. For vSphere installations, copy one of ocp-vsphere-*.yaml files into the $CONFIG_DIR/config directory. If you also want to install a Cloud Pak, copy one of the cp4*.yaml files.
Example:
mkdir -p $HOME/cpd-config/config
+cp sample-configurations/sample-dynamic/config-samples/ocp-vsphere-ocs-nfs.yaml $HOME/cpd-config/config/
+cp sample-configurations/sample-dynamic/config-samples/cp4d-471.yaml $HOME/cpd-config/config/
+Cloud Pak Deployer uses the status directory to log its activities and also to keep track of its running state. For a given environment you're provisioning or destroying, you should always specify the same status directory to avoid contention between different deploy runs.
export CONFIG_DIR=$HOME/cpd-config
+export STATUS_DIR=$HOME/cpd-status
+CONFIG_DIR: Directory that holds the configuration, it must have a config subdirectory which contains the configuration yaml files.STATUS_DIR: The directory where the Cloud Pak Deployer keeps all status information and logs files.If the deployer configuration is kept on GitHub, follow the instructions in GitHub configuration.
For special configuration with defaults and dynamic variables, refer to Advanced configuration.
In order to successfully install OpenShift on vSphere infrastructure, the following pre-requisites must have been met.
| Pre-requisite | Description |
|---|---|
| Red Hat pull secret | A pull secret is required to download and install OpenShift. See Acquire pull secret |
| IBM Entitlement key | When instaling an IBM Cloud Pak, you need an IBM entitlement key. See Acquire IBM Cloud Pak entitlement key |
| vSphere credentials | The OpenShift IPI installer requires vSphere credentials to create VMs and storage |
| Firewall rules | The OpenShift cluster's API server on port 6443 and application server on port 443 must be reachable. |
| Whitelisted URLs | The OpenShift and Cloud Pak download locations and registry must be accessible from the vSphere infrastructure. See Whitelisted locations |
| DHCP | When provisioning new VMs, IP addresses must be automatically assigned through DHCP |
| DNS | A DNS server that will resolve the OpenShift API server and applications is required. See DNS configuration |
| Time server | A time server to synchronize the time must be available in the network and configured through the DHCP server |
There are also some optional settings, dependent on the specifics of the installation:
| Pre-requisite | Description |
|---|---|
| Bastion server | It can be useful to have a bastion/installation server to run the deployer. This (virtual) server must reside within the vSphere network |
| NFS details | If an NFS server is used for storage, it must be reacheable (firewall) and no_root_squash must be set |
| Private registry | If the installation must use a private registry for the Cloud Pak installation, it must be available and credentials shared |
| Certificates | If the Cloud Pak URL must have a CA-signed certificate, the key, certificate and CA bundle must be available at instlalation time |
| Load balancer | The OpenShift IPI install creates 2 VIPs and takes care of the routing to the services. In some implementations, a load balancer provided by the infrastructure team is preferred. This load balancer must be configured externally |
During the provisioning and configuration process, the deployer needs access to the OpenShift API and the ingress server for which the IP addresses are specified in the openshift object.
Ensure that the DNS server has the following entries:
api.openshift_name.domain_name → Point to the api_vip address configured in the openshift object*.apps.openshift_name.domain_name → Point to the ingress_vip address configured in the openshift objectIf you do not configure the DNS entries upfront, the deployer will still run and it will "spoof" the required entries in the container's /etc/hosts file. However to be able to connect to OpenShift and access the Cloud Pak, the DNS entries are required.
In order for the Cloud Pak Deployer to create the infrastructure and deploy the IBM Cloud Pak, it must have provisioning access to vSphere and it needs the vSphere user and password. The user must have permissions to create virtual machines.
export VSPHERE_USER=your_vsphere_user
+export VSPHERE_PASSWORD=password_of_the_vsphere_user
+VSPHERE_USER: This is the user name of the vSphere user, often this is something like admin@vsphere.localVSPHERE_PASSWORD: The password of the vSphere user. Be careful with special characters like $, ! as they are not accepted by the IPI provisioning of OpenShiftIf you want to pull the Cloud Pak images from the entitled registry (i.e. an online install), or if you want to mirror the images to your private registry, you need to download the entitlement key. You can skip this step if you're installing from a private registry and all Cloud Pak images have already been downloaded to the private registry.
Warning
As stated for the API key, you can choose to download the entitlement key to a file. However, when we reference the entitlement key, we mean the 80+ character string that is displayed, not the file.
To install OpenShift you need an OpenShift pull secret which holds your entitlement.
/tmp/ocp_pullsecret.jsonTo obtain access to the OpenShift nodes post-installation, you will need to specify the public SSH key of your server; typically this is ~/.ssh/id_rsa.pub, where ~ is the home directory of your user. If you don't have an SSH key-pair yet, you can generate one using the steps documented here: https://cloud.ibm.com/docs/ssh-keys?topic=ssh-keys-generating-and-using-ssh-keys-for-remote-host-authentication#generating-ssh-keys-on-linux. Alternatively, deployer can generate SSH key-pair automatically if credential ocp-ssh-pub-key is not in the vault.
If you want the Cloud Pak images to be pulled from the entitled registry, set the Cloud Pak entitlement key.
export CP_ENTITLEMENT_KEY=your_cp_entitlement_key
+CP_ENTITLEMENT_KEY: This is the entitlement key you acquired as per the instructions above, this is a 80+ character string. You don't need to set this environment variable when you install the Cloud Pak(s) from a private registryYou need to store the OpenShift pull secret in the vault so that the deployer has access to it.
./cp-deploy.sh vault set \
+ --vault-secret ocp-pullsecret \
+ --vault-secret-file /tmp/ocp_pullsecret.json
+If you want to use your SSH key to access nodes in the cluster, set the Vault secret with the public SSH key.
./cp-deploy.sh vault set \
+ --vault-secret ocp-ssh-pub-key \
+ --vault-secret-file ~/.ssh/id_rsa.pub
+In some cases, download of the cloudctl and cpd-cli clients from @IBM will fail because GitHub limits the number of API calls from non-authenticated clients. You can remediate this issue by creating a Personal Access Token on github.com and creating a secret in the vault.
./cp-deploy.sh vault set -vs github-ibm-pat=<your PAT>
+Alternatively, you can set the secret by adding -vs github-ibm-pat=<your PAT> to the ./cp-deploy.sh env apply command.
If you only want to validate the configuration, you can run the dpeloyer with the --check-only argument. This will run the first stage to validate variables and vault secrets and then execute the generators.
./cp-deploy.sh env apply --check-only --accept-all-licenses
+To run the container using a local configuration input directory and a data directory where temporary and state is kept, use the example below. If you don't specify the status directory, the deployer will automatically create a temporary directory. Please note that the status directory will also hold secrets if you have configured a flat file vault. If you lose the directory, you will not be able to make changes to the configuration and adjust the deployment. It is best to specify a permanent directory that you can reuse later. If you specify an existing directory the current user must be the owner of the directory. Failing to do so may cause the container to fail with insufficient permissions.
./cp-deploy.sh env apply --accept-all-licenses
+You can also specify extra variables such as env_id to override the names of the objects referenced in the .yaml configuration files as {{ env_id }}-xxxx. For more information about the extra (dynamic) variables, see advanced configuration.
The --accept-all-licenses flag is optional and confirms that you accept all licenses of the installed cartridges and instances. Licenses must be either accepted in the configuration files or at the command line.
When running the command, the container will start as a daemon and the command will tail-follow the logs. You can press Ctrl-C at any time to interrupt the logging but the container will continue to run in the background.
You can return to view the logs as follows:
./cp-deploy.sh env logs
+Deploying the infrastructure, preparing OpenShift and installing the Cloud Pak will take a long time, typically between 1-5 hours,dependent on which Cloud Pak cartridges you configured. For estimated duration of the steps, refer to Timings.
If you need to interrupt the automation, use CTRL-C to stop the logging output and then use:
./cp-deploy.sh env kill
+If the Cloud Pak Deployer fails, for example because certain infrastructure components are temporarily not available, fix the cause if needed and then just re-run it with the same CONFIG_DIR and STATUS_DIR as well extra variables. The provisioning process has been designed to be idempotent and it will not redo actions that have already completed successfully.
Once the process has finished, it will output the URLs by which you can access the deployed Cloud Pak. You can also find this information under the cloud-paks directory in the status directory you specified.
To retrieve the Cloud Pak URL(s):
cat $STATUS_DIR/cloud-paks/*
+This will show the Cloud Pak URLs:
Cloud Pak for Data URL for cluster pluto-01 and project cpd (domain name specified was example.com):
+https://cpd-cpd.apps.pluto-01.example.com
+The admin password can be retrieved from the vault as follows:
List the secrets in the vault:
./cp-deploy.sh vault list
+This will show something similar to the following:
Secret list for group sample:
+- vsphere-user
+- vsphere-password
+- ocp-pullsecret
+- ocp-ssh-pub-key
+- ibm_cp_entitlement_key
+- sample-kubeadmin-password
+- cp4d_admin_cpd_demo
+You can then retrieve the Cloud Pak for Data admin password like this:
./cp-deploy.sh vault get --vault-secret cp4d_admin_cpd_demo
+PLAY [Secrets] *****************************************************************
+included: /cloud-pak-deployer/automation-roles/99-generic/vault/vault-get-secret/tasks/get-secret-file.yml for localhost
+cp4d_admin_zen_sample_sample: gelGKrcgaLatBsnAdMEbmLwGr
+You can find examples of a couple of typical changes you may want to do here: Post-run changes.
When deploying self-managed OpenShift on AWS, the compute nodes are represented as one or more OpenShift MachineSets. If your cluster is deployed in a single zone, there will be 1 MachineSet which defines the number and type of ec2 instances created in the AWS account. For multi-zone clusters, there will be 3 MachineSets.
MachineSet🔗Below is an example of a compute node (worker) MachineSet created by the OpenShift installer. The instance type defines the node that is spun up, with 32 vCPUs and 128 GB of memory.
apiVersion: machine.openshift.io/v1beta1
+kind: MachineSet
+metadata:
+ annotations:
+ capacity.cluster-autoscaler.kubernetes.io/labels: kubernetes.io/arch=amd64
+ machine.openshift.io/GPU: '0'
+ machine.openshift.io/memoryMb: '131072'
+ machine.openshift.io/vCPU: '32'
+ resourceVersion: '22985'
+ name: fk-aws-sts-2th7t-worker-us-east-1a
+ uid: be5a9880-eaa0-4054-a77f-e5c4432eb51f
+ creationTimestamp: '2024-10-16T20:30:43Z'
+ generation: 1
+ managedFields:
+ - apiVersion: machine.openshift.io/v1beta1
+ fieldsType: FieldsV1
+ fieldsV1:
+ 'f:metadata':
+ 'f:labels':
+ .: {}
+ 'f:machine.openshift.io/cluster-api-cluster': {}
+ 'f:spec':
+ .: {}
+ 'f:replicas': {}
+ 'f:selector': {}
+ 'f:template':
+ .: {}
+ 'f:metadata':
+ .: {}
+ 'f:labels':
+ .: {}
+ 'f:machine.openshift.io/cluster-api-cluster': {}
+ 'f:machine.openshift.io/cluster-api-machine-role': {}
+ 'f:machine.openshift.io/cluster-api-machine-type': {}
+ 'f:machine.openshift.io/cluster-api-machineset': {}
+ 'f:spec':
+ .: {}
+ 'f:lifecycleHooks': {}
+ 'f:metadata': {}
+ 'f:providerSpec':
+ .: {}
+ 'f:value':
+ 'f:instanceType': {}
+ 'f:metadata':
+ .: {}
+ 'f:creationTimestamp': {}
+ 'f:blockDevices': {}
+ 'f:kind': {}
+ 'f:securityGroups': {}
+ 'f:deviceIndex': {}
+ 'f:ami':
+ .: {}
+ 'f:id': {}
+ 'f:metadataServiceOptions': {}
+ 'f:tags': {}
+ .: {}
+ 'f:placement':
+ .: {}
+ 'f:availabilityZone': {}
+ 'f:region': {}
+ 'f:subnet':
+ .: {}
+ 'f:filters': {}
+ 'f:apiVersion': {}
+ 'f:iamInstanceProfile':
+ .: {}
+ 'f:id': {}
+ 'f:credentialsSecret':
+ .: {}
+ 'f:name': {}
+ 'f:userDataSecret':
+ .: {}
+ 'f:name': {}
+ manager: cluster-bootstrap
+ operation: Update
+ time: '2024-10-16T20:30:43Z'
+ - apiVersion: machine.openshift.io/v1beta1
+ fieldsType: FieldsV1
+ fieldsV1:
+ 'f:status': {}
+ manager: cluster-bootstrap
+ operation: Update
+ subresource: status
+ time: '2024-10-16T20:30:43Z'
+ - apiVersion: machine.openshift.io/v1beta1
+ fieldsType: FieldsV1
+ fieldsV1:
+ 'f:metadata':
+ 'f:annotations':
+ .: {}
+ 'f:capacity.cluster-autoscaler.kubernetes.io/labels': {}
+ 'f:machine.openshift.io/GPU': {}
+ 'f:machine.openshift.io/memoryMb': {}
+ 'f:machine.openshift.io/vCPU': {}
+ manager: machine-controller-manager
+ operation: Update
+ time: '2024-10-16T20:35:26Z'
+ - apiVersion: machine.openshift.io/v1beta1
+ fieldsType: FieldsV1
+ fieldsV1:
+ 'f:status':
+ 'f:availableReplicas': {}
+ 'f:fullyLabeledReplicas': {}
+ 'f:observedGeneration': {}
+ 'f:readyReplicas': {}
+ 'f:replicas': {}
+ manager: machineset-controller
+ operation: Update
+ subresource: status
+ time: '2024-10-16T20:41:50Z'
+ namespace: openshift-machine-api
+ labels:
+ machine.openshift.io/cluster-api-cluster: fk-aws-sts-2th7t
+spec:
+ replicas: 3
+ selector:
+ matchLabels:
+ machine.openshift.io/cluster-api-cluster: fk-aws-sts-2th7t
+ machine.openshift.io/cluster-api-machineset: fk-aws-sts-2th7t-worker-us-east-1a
+ template:
+ metadata:
+ labels:
+ machine.openshift.io/cluster-api-cluster: fk-aws-sts-2th7t
+ machine.openshift.io/cluster-api-machine-role: worker
+ machine.openshift.io/cluster-api-machine-type: worker
+ machine.openshift.io/cluster-api-machineset: fk-aws-sts-2th7t-worker-us-east-1a
+ spec:
+ lifecycleHooks: {}
+ metadata: {}
+ providerSpec:
+ value:
+ userDataSecret:
+ name: worker-user-data
+ placement:
+ availabilityZone: us-east-1a
+ region: us-east-1
+ credentialsSecret:
+ name: aws-cloud-credentials
+ instanceType: m5.8xlarge
+ metadata:
+ creationTimestamp: null
+ blockDevices:
+ - ebs:
+ encrypted: true
+ iops: 0
+ kmsKey:
+ arn: ''
+ volumeSize: 120
+ volumeType: gp3
+ securityGroups:
+ - filters:
+ - name: 'tag:Name'
+ values:
+ - fk-aws-sts-2th7t-worker-sg
+ kind: AWSMachineProviderConfig
+ metadataServiceOptions: {}
+ tags:
+ - name: kubernetes.io/cluster/fk-aws-sts-2th7t
+ value: owned
+ deviceIndex: 0
+ ami:
+ id: ami-0d653d86d4113326a
+ subnet:
+ filters:
+ - name: 'tag:Name'
+ values:
+ - fk-aws-sts-2th7t-private-us-east-1a
+ apiVersion: machine.openshift.io/v1beta1
+ iamInstanceProfile:
+ id: fk-aws-sts-2th7t-worker-profile
+status:
+ availableReplicas: 3
+ fullyLabeledReplicas: 3
+ observedGeneration: 1
+ readyReplicas: 3
+ replicas: 3
+MachineSet🔗To create a GPU MachineSet in the same region, copy the yaml into your favourite text editor and remove all the unnecessary properties. Below is an example of the resulting yaml, highlighting the items that have changed to define the GPU node(s). In this example there is only 1 GPU node of type g6e.8xlarge. Only the highlighted properties should be changed.
1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 +10 +11 +12 +13 +14 +15 +16 +17 +18 +19 +20 +21 +22 +23 +24 +25 +26 +27 +28 +29 +30 +31 +32 +33 +34 +35 +36 +37 +38 +39 +40 +41 +42 +43 +44 +45 +46 +47 +48 +49 +50 +51 +52 +53 +54 +55 +56 +57 +58 +59 +60 +61 +62 | |
MachineSet🔗Once ready, do the following to create the MachineSet:
The MachineSet will create Machine CRs in the openshift-machines project. If the instance type is available in the selected AWS region and there is enough capacity, the AWS instance(s) will be created and after a 5-10 minutes, it/they will appear as nodes in the OpenShift cluster.
If you want to change the deployed configuration, you can just update the configuration files and re-run the deployer. Make sure that you use the same input configuration and status directories and also the env_id if you specified one, otherwise deployment may fail.
Below are a couple of examples of post-run changes you may want to do.
When initially installed, the Cloud Pak Deployer will generate a strong password for the Cloud Pak for Data admin user (or cpadmin if you have selected to use Foundational Services IAM). If you want to change the password afterwards, you can do this from the Cloud Pak for Data user interface, but this means that the deployer will no longer be able to make changes to the Cloud Pak for Data configuration.
If you have updated the admin password from the UI, please make sure you also update the secret in the vault.
First, list the secrets in the vault:
./cp-deploy.sh vault list
+This will show something similar to the following:
Secret list for group sample:
+- ibm_cp_entitlement_key
+- sample-provision-ssh-key
+- sample-provision-ssh-pub-key
+- sample-terraform-tfstate
+- cp4d_admin_zen_sample_sample
+Then, update the password:
./cp-deploy.sh vault set -vs cp4d_admin_zen_sample_sample -vsv "my Really Sec3re Passw0rd"
+Finally, run the deployer again. It will make the necessary changes to the OpenShift secret and check that the admin (or cpadmin) user can log in. In this case you can speed up the process via the --skip-infra flag.
./cp-deploy.sh env apply --skip-infra [--accept-all-liceneses]
+watsonx.ai requires GPUs to run and tune the foundation models. Deployer currently does not provision these GPU nodes, but you can add them manually from the OpenShift console.
For adding GPU nodes on AWS infrastructure, refer to Add GPUs to self-managed OpenShift on AWS.
Sometimes you may need to access the OpenShift cluster using the OpenShift client. For convenience we have made the oc command available in the Cloud Pak Deployer and you can start exploring the current OpenShift cluster immediately without having to install the client on your own workstation.
Make sure you have set the CONFIG_DIR and STATUS_DIR environment variables to the same values when you ran the env apply command. This will ensure that the oc command will access the OpenShift cluster(s) of that configuration.
If you have not run the deployer yet and do not intend to install any Cloud Paks, but you do want to access the OpenShift cluster from the command line to check or prepare items, run the deployer with the --skip-cp-install flag.
./cp-deploy.sh env apply --skip-cp-install
+Deployer will check the configuration, download clients, attempt to login to OpenShift and prepare the OpenShift cluster with the global pull secret and (for Cloud Pak for Data) node settings. After that the deployer will finish without installing any Cloud Pak.
./cp-deploy.sh env cmd
+You should see something like this:
-------------------------------------------------------------------------------
+Entering Cloud Pak Deployer command line in a container.
+Use the "exit" command to leave the container and return to the hosting server.
+-------------------------------------------------------------------------------
+Installing OpenShift client
+Current OpenShift context: cpd
+Now, you can check the OpenShift cluster version:
oc get clusterversion
+NAME VERSION AVAILABLE PROGRESSING SINCE STATUS
+version 4.8.14 True False 2d3h Cluster version is 4.8.14
+Or, display the list of OpenShift projects:
oc get projects | grep -v openshift-
+NAME DISPLAY NAME STATUS
+calico-system Active
+default Active
+ibm-cert-store Active
+ibm-odf-validation-webhook Active
+ibm-system Active
+kube-node-lease Active
+kube-public Active
+kube-system Active
+openshift Active
+services Active
+tigera-operator Active
+cpd Active
+Once finished, exit out of the container.
exit
+If you have previously used the Cloud Pak Deployer to create assets, you can destroy the assets with the same command.
Info
Currently, destroy is only implemented for OpenShift clusters on IBM Cloud ROKS, AWS and Azure, and for Cloud Pak for Data on an existing OpenShift cluster.
Optional: set environment variables for deployer config and status directories. If not specified, respectively $HOME/cpd-config and $HOME/cpd-status will be used.
export STATUS_DIR=$HOME/cpd-status
+export CONFIG_DIR=$HOME/cpd-config
+STATUS_DIR: The directory where the Cloud Pak Deployer keeps all status information and log files. Please note that if you have chosen to use a File Vault, the directory specified must be the one you used when you created the environmentCONFIG_DIR: Directory that holds the configuration. This must be the same directory you used when you created the environmentexport IBM_CLOUD_API_KEY=your_api_key
+Optional: set environment variables for deployer config and status directories. If not specified, respectively $HOME/cpd-config and $HOME/cpd-status will be used.
export STATUS_DIR=$HOME/cpd-status
+export CONFIG_DIR=$HOME/cpd-config
+IBM_CLOUD_API_KEY: This is the API key you generated using your IBM Cloud account, this is a 40+ character stringSTATUS_DIR: The directory where the Cloud Pak Deployer keeps all status information and log files. Please note that if you have chosen to use a File Vault, the directory specified must be the one you used when you created the environmentCONFIG_DIR: Directory that holds the configuration. This must be the same directory you used when you created the environmentWe assume that the vault already holds the mandatory secrets for AWS Access Key, Secret Access Key and ROSA login token.
export STATUS_DIR=$HOME/cpd-status
+export CONFIG_DIR=$HOME/cpd-config
+STATUS_DIR: The directory where the Cloud Pak Deployer keeps all status information and log files. Please note that if you have chosen to use a File Vault, the directory specified must be the one you used when you created the environmentCONFIG_DIR: Directory that holds the configuration. This must be the same directory you used when you created the environmentWe assume that the vault already holds the mandatory secrets for Azure - Service principal id and its password, tenant id and ARO login token.
export STATUS_DIR=$HOME/cpd-status
+export CONFIG_DIR=$HOME/cpd-config
+STATUS_DIR: The directory where the Cloud Pak Deployer keeps all status information and log files. Please note that if you have chosen to use a File Vault, the directory specified must be the one you used when you created the environmentCONFIG_DIR: Directory that holds the configuration. This must be the same directory you used when you created the environment./cp-deploy.sh env destroy --confirm-destroy
+Please ensure you specify the same extra (dynamic) variables that you used when you ran the env apply command.
When running the command, the container will start as a daemon and the command will tail-follow the logs. You can press Ctrl-C at any time to interrupt the logging but the container will continue to run in the background.
You can return to view the logs as follows:
./cp-deploy.sh env logs
+If you need to interrupt the process, use CTRL-C to stop the logging output and then use:
./cp-deploy.sh env kill
+Once the process has finished successfully, you can delete the status directory.
Defines the Cloud Pak(s) which is/are layed out on the OpenShift cluster, typically in one or more OpenShift projects. The Cloud Pak definition represents the instance users connect to and which is responsible for managing the functional capabilities installed within the application.
cp4d🔗Defines the Cloud Pak for Data instances to be configured on the OpenShift cluster(s).
cp4d:
+- project: cpd
+ openshift_cluster_name: sample
+ cp4d_version: 4.7.3
+ use_fs_iam: False
+ change_node_settings: True
+ db2u_limited_privileges: False
+ accept_licenses: False
+ openshift_storage_name: nfs-storage
+ cp4d_entitlement:
+ - cpd-enterprise
+ # - cpd-standard
+ # - cognos-analytics
+ - data-product-hub
+ # - datastage
+ # - ikc-premium
+ # - ikc-standard
+ # - openpages
+ # - planning-analytics
+ # - product-master
+ # - speech-to-text
+ # - text-to-speech
+ - watson-assistant
+ # - watson-discovery
+ # - watsonx-ai
+ # - watsonx-code-assistant-ansible
+ # - watsonx-code-assistant-z
+ # - watsonx-data
+ # - watsonx-gov-mm
+ # - watsonx-gov-rc
+ # - watsonx-orchestrate
+ cp4d_production_license: True
+ state: installed
+
+ cartridges:
+ - name: cpfs
+ - name: cpd_platform
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| project | Name of the OpenShift project of the Cloud Pak for Data instance | Yes | |
| openshift_cluster_name | Name of the OpenShift cluster | Yes, inferred from openshift | Existing openshift cluster |
| cp4d_version | Cloud Pak for Data version to install, this will determine the version for all cartridges that do not specify a version | Yes | 4.x.x |
| sequential_install | Deprecated property | No | True (default), False |
| use_fs_iam | If set to True the deployer will enable Foundational Services IAM for authentication | No | False (default), True |
| use_cp_alt_repo | When set to False, deployer will use use the alternative repo specified in cp_alt_repo resource | No | True (default), False |
| change_node_settings | Controls whether the node settings using the machine configs will be applied onto the OpenShift cluster. | No | True, False |
| db2u_limited_privileges | Depicts whether Db2U containers run with limited privileges. If they do (True), Deployer will create KubeletConfig and Tuned OpenShift resources as per the documentation. | No | False (default), True |
| accept_licenses | Set to 'True' to accept Cloud Pak licenses. Alternatively the --accept-all-licenses can be used for the cp-deploy.sh command | No | True, False (default) |
| cp4d_entitlement | Set to cpd-enterprise, cpd-standard, watsonx-data, watsonx-ai, watsonx-gov-mm, watsonx-gov-rc, dependent on the deployed license, multiple entitlements can be specified | No | For valid values, refer to product documentation |
| cp4d_production_license | Whether the Cloud Pak for Data is a production license | No | True (default), False |
| state | Indicated whether Cloud Pak for Data must be installed or removed | No | installed (default), removed |
| image_registry_name | When using private registry, specify name of image_registry | No | |
| openshift_storage_name | References an openshift_storage element in the OpenShift cluster that was defined for this Cloud Pak for Data instance. The name must exist under `openshift.[openshift_cluster_name].openshift_storage. | No, inferred from openshift->openshift_storage | |
| cartridges | List of cartridges to install for this Cloud Pak for Data instance. See Cloud Pak for Data cartridges for more details | Yes |
cp4i🔗Defines the Cloud Pak for Integration installation to be configured on the OpenShift cluster(s).
cp4i:
+- project: cp4i
+ openshift_cluster_name: {{ env_id }}
+ openshift_storage_name: nfs-rook-ceph
+ cp4i_version: 2021.4.1
+ accept_licenses: False
+ use_top_level_operator: False
+ top_level_operator_channel: v1.5
+ top_level_operator_case_version: 2.5.0
+ operators_in_all_namespaces: True
+
+ instances:
+ - name: integration-navigator
+ type: platform-navigator
+ license: L-RJON-C7QG3S
+ channel: v5.2
+ case_version: 1.5.0
+The immediate content of the cp4i object is actually a list of OpenShift projects (namespaces). There can be more than one project and instances can be created in separate projects.
cp4i:
+- project: cp4i
+ ...
+
+- project: cp4i-ace
+ ...
+
+- project: cp4i-apic
+ ...
+Before you run the Cloud Pak Deployer be sure that the correct operator channels are defined for the selected instance types. Some products require a license ID, please check the documentation of each product for the correct license. If you decide to use CASE files instead of the IBM Operator Catalog (more on that below) make sure that you selected the correct CASE versions - please refer: https://github.com/IBM/cloud-pak/tree/master/repo/case
The following properties are defined on the project level:
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| project | The name of the OpenShift project that will be created and used for the installation of the defined instances. | Yes | |
| openshift_cluster_name | Dynamically defined form the env_id parameter during the execution. | Yes, inferred from openshift | Existing openshift cluster |
| openshift_storage_name | Reference to the storage definition that exists in the openshift object (please see above). The definition must include the class name of the file storage type and the class name of the block storage type. | No, inferred from openshift->openshift_storage | |
| cp4i_version | The version of the Cloud Pak for Integration (e.g. 2021.4.1) | Yes | |
| use_case_files | The property defines if the CASE files are used for installation. If it is True then the operator catalogs are created from the CASE files. If it is False, the IBM Operator Catalog from the entitled registry is used. | No | True, False (default) |
| accept_licenses | Set to True to accept Cloud Pak licenses. Alternatively the --accept-all-licenses can be used for the cp-deploy.sh command | Yes | True, False |
| use_top_level_operator | If it is True then the CP4I top-level operator that installs all other operators is used. Otherwise, only the operators for the selected instance types are installed. | No | True, False (default) |
| top_level_operator_channel | Needed if the use_top_level_operator is True otherwise, it is ignored. Specifies the channel of the top-level operator. | No | |
| top_level_operator_case_version | Needed if the use_top_level_operator is True otherwise, it is ignored. Specifies the CASE package version of the top-level operator. | No | |
| operators_in_all_namespaces | It defines whether the operators are visible in all namespaces or just in the specific namespace where they are needed. | No | True, False (default) |
| instances | List of the instances that are going to be created (please see below). | Yes |
Warning
Despite the properties use_case_files, use_top_level_operator and operators_in_all_namespaces are defined as optional, they are actually crucial for the way of execution of the installation process. If any of them is omitted, it is assumed that the default False value is used. If none of them exists, it means that all are False. In this case, it means that the IBM Operator Catalog is used and only the needed operators for specified instance types are installed in the specific namespace.
The instance property contains one or more instances definitions. Each instance must have a unique name. There can be more the one instance of the same type.
For each instance definition, an instance type must be specified. We selected the type names that are as much as possible similar to the naming convention used in the Platform Navigator use interface. The following table shows all existing types:
| Instance type | Description/Product name |
|---|---|
| platform-navigator | Platform Navigator |
| api-management | IBM API Connect |
| automation-assets | Automation assets a.k.a Asset repo |
| enterprise-gateway | IBM Data Power |
| event-endpoint-management | Event endpoint manager - managing asynchronous APIs |
| event-streams | IBM Event Streams - Kafka |
| high-speed-transfer-server | Aspera HSTS |
| integration-dashboard | IBM App Connect Integration Dashboard |
| integration-design | IBM App Connect Designer |
| integration-tracing | Operations Dashboard |
| messaging | IBM MQ |
The Platform Navigator is defined as one of the instance types. There is typically only one instance of it. The exception would be an installation in two or more completely separate namespaces (see the CP4I documentation). Special attention is paid to the installation of the Navigator. The Cloud Pak Deployer will install the Navigator instance first, before any other instance, and it will wait until the instance is ready (this could take up to 45 minutes).
When the installation is completed, you will find the admin user password in the status/cloud-paks/cp4i-
| Property | Description | Sample value for 2021.4.1 |
|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | |
| type | It must be platform-navigator | |
| license | License ID | L-RJON-C7QG3S |
| channel | Subscription channel | v5.2 |
| case_version | CASE version | 1.5.0 |
| Property | Description | Sample value for 2021.4.1 |
|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | |
| type | It must be api-management | |
| license | License ID | L-RJON-C7BJ42 |
| version | Version of API Connect | 10.0.4.0 |
| channel | Subscription channel | v2.4 |
| case_version | CASE version | 3.0.5 |
| Property | Description | Sample value for 2021.4.1 |
|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | |
| type | It must be automation-assets | |
| license | License ID | L-PNAA-C68928 |
| version | Version of Asset repo | 2021.4.1-2 |
| channel | Subscription channel | v1.4 |
| case_version | CASE version | 1.4.2 |
| Property | Description | Sample value for 2021.4.1 |
|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | |
| type | It must be enterprise-gateway | |
| admin_password_secret | The name of the secret where admin password is stored. The default name is used if you leave it empty. | |
| license | License ID | L-RJON-BYDR3Q |
| version | Version of Data Power | 10.0-cd |
| channel | Subscription channel | v1.5 |
| case_version | CASE version | 1.5.0 |
| Property | Description | Sample value for 2021.4.1 |
|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | |
| type | It must be event-endpoint-management | |
| license | License ID | L-RJON-C7BJ42 |
| version | Version of Event endpoint manager | 10.0.4.0 |
| channel | Subscription channel | v2.4 |
| case_version | CASE version | 3.0.5 |
| Property | Description | Sample value for 2021.4.1 |
|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | |
| type | It must be event-streams | |
| version | Version of Event streams | 10.5.0 |
| channel | Subscription channel | v2.5 |
| case_version | CASE version | 1.5.2 |
| Property | Description | Sample value for 2021.4.1 |
|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | |
| type | It must be high-speed-transfer-server | |
| aspera_key | A license key for the Aspera software | |
| redis_version | Version of the Redis database | 5.0.9 |
| version | Version of Aspera HSTS | 4.0.0 |
| channel | Subscription channel | v1.4 |
| case_version | CASE version | 1.4.0 |
| Property | Description | Sample value for 2021.4.1 |
|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | |
| type | It must be integration-dashboard | |
| license | License ID | L-APEH-C79J9U |
| version | Version of IBM App Connect | 12.0 |
| channel | Subscription channel | v3.1 |
| case_version | CASE version | 3.1.0 |
| Property | Description | Sample value for 2021.4.1 |
|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | |
| type | It must be integration-design | |
| license | License ID | L-KSBM-C87FU2 |
| version | Version of IBM App Connect | 12.0 |
| channel | Subscription channel | v3.1 |
| case_version | CASE version | 3.1.0 |
| Property | Description | Sample value for 2021.4.1 |
|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | |
| type | It must be integration-tracing | |
| version | Version of Integration tracing | 2021.4.1-2 |
| channel | Subscription channel | v2.5 |
| case_version | CASE version | 2.5.2 |
| Property | Description | Sample value for 2021.4.1 |
|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | |
| type | It must be messaging | |
| queue_manager_name | The name of the initial queue. Default is QUICKSTART | |
| license | License ID | L-RJON-C7QG3S |
| version | Version of IBM MQ | 9.2.4.0-r1 |
| channel | Subscription channel | v1.7 |
| case_version | CASE version | 1.7.0 |
cp4waiops🔗Defines the Cloud Pak for Watson AIOps installation to be configured on the OpenShift cluster(s). The following instances can be installed by the deployer:
Aside from the base install, the deployer can also install ready-to-use demos for each of the instances
cp4waiops:
+- project: cp4waiops
+ openshift_cluster_name: "{{ env_id }}"
+ openshift_storage_name: auto-storage
+ accept_licenses: False
+
+ instances:
+ - name: cp4waiops-aimanager
+ kind: AIManager
+ install: true
+ ...
+The following properties are defined on the project level:
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| project | The name of the OpenShift project that will be created and used for the installation of the defined instances. | Yes | |
| openshift_cluster_name | Dynamically defined form the env_id parameter during the execution. | No, only if mutiple OpenShift clusters defined | Existing openshift cluster |
| openshift_storage_name | Reference to the storage definition that exists in the openshift object (please see above). | No, inferred from openshift->openshift_storage | |
| accept_licenses | Set to True to accept Cloud Pak licenses. Alternatively the --accept-all-licenses can be used for the cp-deploy.sh command | Yes | True, False |
The project that is specified at the cp4waiops level defines the OpenShift project into which the instances of each of the services will be installed. Below is a list of instance "kinds" that can be installed. For every "service instance" there can also be a "demo content" entry to prepare the demo content for the capability.
instances:
+ - name: cp4waiops-aimanager
+ kind: AIManager
+ install: true
+
+ waiops_size: small
+ custom_size_file: none
+ waiops_name: ibm-cp-watson-aiops
+ subscription_channel: v3.6
+ freeze_catalog: false
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | Yes | |
| kind | Service kind to install | Yes | AIManager |
| install | Must the service be installed? | Yes | true, false |
| waiops_size | Size of the install | Yes | small, tall, custom |
| custom_size_file | Name of the file holding the custom sizes if waiops_size is custom | No | |
| waiops_name | Name of the CP4WAIOPS instance | Yes | |
| subscription_channel | Subscription channel of the operator | Yes | |
| freeze_catalog | Freeze the version of the catalog source? | Yes | false, true |
| case_install | Must AI manager be installed via case files? | No | false, true |
| case_github_url | GitHub URL to download case file | Yes if case_install is true | |
| case_name | Name of the case file | Yes if case_install is true | |
| case_version | Version of the case file to download | Yes if case_install is true | |
| case_inventory_setup | Case file operation to run for this service | Yes if case_install is true | cpwaiopsSetup |
instances:
+ - name: cp4waiops-aimanager-demo-content
+ kind: AIManagerDemoContent
+ install: true
+ ...
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | Yes | |
| kind | Service kind to install | Yes | AIManagerDemoContent |
| install | Must the content be installed? | Yes | true, false |
See sample config for remainder of properties.
instances:
+ - name: cp4waiops-eventmanager
+ kind: EventManager
+ install: true
+ subscription_channel: v1.11
+ starting_csv: noi.v1.7.0
+ noi_version: 1.6.6
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | Yes | |
| kind | Service kind to install | Yes | EventManager |
| install | Must the service be installed? | Yes | true, false |
| subscription_channel | Subscription channel of the operator | Yes | |
| starting_csv | Starting Cluster Server Version | Yes | |
| noi_version | Version of noi | Yes |
instances:
+ - name: cp4waiops-eventmanager
+ kind: EventManagerDemoContent
+ install: true
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | Yes | |
| kind | Service kind to install | Yes | EventManagerDemoContent |
| install | Must the content be installed? | Yes | true, false |
instances:
+ - name: cp4waiops-infrastructure-management
+ kind: InfrastructureManagement
+ install: false
+ subscription_channel: v3.5
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | Yes | |
| kind | Service kind to install | Yes | InfrastructureManagement |
| install | Must the service be installed? | Yes | true, false |
| subscription_channel | Subscription channel of the operator | Yes |
ElasticSearch, Logstash and Kibana stack.
instances:
+ - name: cp4waiops-elk
+ kind: ELK
+ install: false
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | Yes | |
| kind | Service kind to install | Yes | ELK |
| install | Must the service be installed? | Yes | true, false |
instances:
+ - name: cp4waiops-instana
+ kind: Instana
+ install: true
+ version: 241-0
+
+ sales_key: 'NONE'
+ agent_key: 'NONE'
+
+ instana_admin_user: "admin@instana.local"
+ #instana_admin_pass: 'P4ssw0rd!'
+
+ install_agent: true
+
+ integrate_aimanager: true
+ #integrate_turbonomic: true
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | Yes | |
| kind | Service kind to install | Yes | Instana |
| install | Must the service be installed? | Yes | true, false |
| version | Version of Instana to install | No | |
| sales_key | License key to be configured | No | |
| agent_key | License key for agent to be configured | No | |
| instana_admin_user | Instana admin user to be configured | Yes | |
| instana_admin_pass | Instana admin user password to be set (if different from global password) | No | |
| install_agent | Must the Instana agent be installed? | Yes | true, false |
| integrate_aimanager | Must Instana be integrated with AI Manager? | Yes | true, false |
| integrate_turbonomic | Must Instana be integrated with Turbonomic? | No | true, false |
instances:
+ - name: cp4waiops-turbonomic
+ kind: Turbonomic
+ install: true
+ turbo_version: 8.7.0
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | Yes | |
| kind | Service kind to install | Yes | Turbonomic |
| install | Must the service be installed? | Yes | true, false |
| turbo_version | Version of Turbonomic to install | Yes |
instances:
+ - name: cp4waiops-turbonomic-demo-content
+ kind: TurbonomicDemoContent
+ install: true
+ #turbo_admin_password: P4ssw0rd!
+ create_user: false
+ demo_user: demo
+ #turbo_demo_password: P4ssw0rd!
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Unique name within the cluster using only lowercase alphanumerics and "-" | Yes | |
| kind | Service kind to install | Yes | TurbonomicDemoContent |
| install | Must the content be installed? | Yes | true, false |
| turbo_admin_pass | Turbonomic admin user password to be set (if different from global password) | No | |
| create_user | Must the demo user be created? | No | false, true |
| demo_user | Name of the demo user | No | |
| turbo_demo_password | Demo user password if different from global password | No |
See sample config for remainder of properties.
cp4ba🔗Defines the Cloud Pak for Business Automation installation to be configured on the OpenShift cluster(s).
See Cloud Pak for Business Automation for additional details.
---
+cp4ba:
+- project: cp4ba
+ collateral_project: cp4ba-collateral
+ openshift_cluster_name: "{{ env_id }}"
+ openshift_storage_name: auto-storage
+ accept_licenses: false
+ state: installed
+ cpfs_profile_size: small # Profile size which affect replicas and resources of Pods of CPFS as per https://www.ibm.com/docs/en/cpfs?topic=operator-hardware-requirements-recommendations-foundational-services
+
+ # Section for Cloud Pak for Business Automation itself
+ cp4ba:
+ # Set to false if you don't want to install (or remove) CP4BA
+ enabled: true # Currently always true
+ profile_size: small # Profile size which affect replicas and resources of Pods as per https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=pcmppd-system-requirements
+ patterns:
+ foundation: # Foundation pattern, always true - https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=deployment-capabilities-production-deployments#concept_c2l_1ks_fnb__foundation
+ optional_components:
+ bas: true # Business Automation Studio (BAS)
+ bai: true # Business Automation Insights (BAI)
+ ae: true # Application Engine (AE)
+ decisions: # Operational Decision Manager (ODM) - https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=deployment-capabilities-production-deployments#concept_c2l_1ks_fnb__odm
+ enabled: true
+ optional_components:
+ decision_center: true # Decision Center (ODM)
+ decision_runner: true # Decision Runner (ODM)
+ decision_server_runtime: true # Decision Server (ODM)
+ # Additional customization for Operational Decision Management
+ # Contents of the following will be merged into ODM part of CP4BA CR yaml file. Arrays are overwritten.
+ cr_custom:
+ spec:
+ odm_configuration:
+ decisionCenter:
+ # Enable support for decision models
+ disabledDecisionModel: false
+ decisions_ads: # Automation Decision Services (ADS) - https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=deployment-capabilities-production-deployments#concept_c2l_1ks_fnb__ads
+ enabled: true
+ optional_components:
+ ads_designer: true # Designer (ADS)
+ ads_runtime: true # Runtime (ADS)
+ gen_ai: # https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=services-configuring-generative-ai-secret
+ apiKey: <watsonx_ai_api_key>
+ authUrl: https://iam.bluemix.net/identity/token
+ mlUrl: https://us-south.ml.cloud.ibm.com
+ projectId: <project_id>
+ content: # FileNet Content Manager (FNCM) - https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=deployment-capabilities-production-deployments#concept_c2l_1ks_fnb__ecm
+ enabled: true
+ optional_components:
+ cmis: true # Content Management Interoperability Services (FNCM - CMIS)
+ css: true # Content Search Services (FNCM - CSS)
+ es: true # External Share (FNCM - ES)
+ tm: true # Task Manager (FNCM - TM)
+ ier: true # IBM Enterprise Records (FNCM - IER)
+ icc4sap: false # IBM Content Collector for SAP (FNCM - ICC4SAP) - Currently not implemented
+ application: # Business Automation Application (BAA) - https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=deployment-capabilities-production-deployments#concept_c2l_1ks_fnb__baa
+ enabled: true
+ optional_components:
+ app_designer: true # App Designer (BAA)
+ ae_data_persistence: true # App Engine data persistence (BAA)
+ document_processing: # Automation Document Processing (ADP) - https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=deployment-capabilities-production-deployments#concept_c2l_1ks_fnb__adp
+ enabled: true
+ optional_components:
+ document_processing_designer: true # Designer (ADP)
+ # Additional customization for Automation Document Processing
+ # Contents of the following will be merged into ADP part of CP4BA CR yaml file. Arrays are overwritten.
+ cr_custom:
+ spec:
+ ca_configuration:
+ ## NB: All config parameters for ADP are described here ==> https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=parameters-automation-document-processing
+ ocrextraction:
+ # [Tech Preview] OCR Engine 2 (IOCR) for ADP - Starts the Watson Document Understanding (WDU) pods to process documents.
+ use_iocr: auto # Allowed values: auto, all, none. Refer to doc for option details: https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=parameters-automation-document-processing#:~:text=ocrextraction.use_iocr
+ deep_learning_object_detection: # When enabled, ca_configuration.deeplearning parameters will be used (ignored otherwise), and deep-learning pods will be deployed to enhance object detection.
+ # If disabled, all training will automatically be done in "fast-training" mode and should finish in less than 10 min.
+ # Warn: If you enable this option and don't select the "fast training" mode in ADP before starting training, training could take hours (or more if you don't have GPUs).
+ # See "Important" note here for usage recommandation on using "fast/deeplarning" training: https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/23.0.2?topic=project-creating-data-extraction-model#:~:text=Training%20takes%20time
+ enabled: true
+ deeplearning: # Only used if deep_learning_object_detection is enabled. Configure usage of GPU-enabled Nodes.
+ gpu_enabled: false # Use GPUs for deeplearning training instead of CPUs.
+ nodelabel_key: nvidia.com/gpu.present
+ nodelabel_value: "true"
+ replica_count: 1 # Controls the number of deep learning pod replicas. NB: The number of GPUs available on your cluster should be ≥ to replica_count.
+ workflow: # Business Automation Workflow (BAW) - https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=deployment-capabilities-production-deployments#concept_c2l_1ks_fnb__baw
+ enabled: true
+ optional_components:
+ baw_authoring: true # Workflow Authoring (BAW) - always keep true if workflow pattern is chosen. BAW Runtime is not implemented.
+ kafka: true # Will install a kafka cluster and enable kafka service for workflow authoring.
+
+ # Section for IBM Process mining
+ pm:
+ # Set to false if you don't want to install (or remove) Process Mining
+ enabled: true
+ # Additional customization for Process Mining
+ # Contents of the following will be merged into PM CR yaml file. Arrays are overwritten.
+ cr_custom:
+ spec:
+ processmining:
+ storage:
+ # Disables redis to spare resources as per https://www.ibm.com/docs/en/process-mining/latest?topic=configurations-custom-resource-definition
+ redis:
+ install: false
+
+ # Section for IBM Robotic Process Automation
+ rpa:
+ # Set to false if you don't want to install (or remove) RPA
+ enabled: true
+ # Additional customization for Robotic Process Automation
+ # Contents of the following will be merged into RPA CR yaml file. Arrays are overwritten.
+ cr_custom:
+ spec:
+ # Configures the NLP provider component of IBM RPA. You can disable it by specifying 0. https://www.ibm.com/docs/en/rpa/latest?topic=platform-configuring-rpa-custom-resources#basic-setup
+ nlp:
+ replicas: 1
+
+ # Set to false if you don't want to install (or remove) CloudBeaver (PostgreSQL, DB2, MSSQL UI)
+ cloudbeaver_enabled: true
+
+ # Set to false if you don't want to install (or remove) Roundcube
+ roundcube_enabled: true
+
+ # Set to false if you don't want to install (or remove) Cerebro
+ cerebro_enabled: true
+
+ # Set to false if you don't want to install (or remove) AKHQ
+ akhq_enabled: true
+
+ # Set to false if you don't want to install (or remove) Mongo Express
+ mongo_express_enabled: true
+
+ # Set to false if you don't want to install (or remove) phpLDAPAdmin
+ phpldapadmin_enabled: true
+
+ # Set to false if you don't want to install (or remove) OpenSearch Dashboards
+ opensearch_dashboards_enabled: true
+The following properties are defined on the project level.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| project | The name of the OpenShift project that will be created and used for the installation of the defined instances. | Yes | Valid OCP project name |
| collateral_project | The name of the OpenShift project that will be created and used for the installation of all collateral (prerequisites and extras). | Yes | Valid OCP project name |
| openshift_cluster_name | Dynamically defined form the env_id parameter during the execution. | No, only if multiple OpenShift clusters defined | Existing openshift cluster |
| openshift_storage_name | Reference to the storage definition that exists in the openshift object (please see above). | No, inferred from openshift->openshift_storage | |
| accept_licenses | Set to true to accept Cloud Pak licenses. Alternatively the --accept-all-licenses can be used for the cp-deploy.sh command | Yes | true, false |
| state | Set to installed to install enabled capabilities, set to removed to remove enabled capabilities. | Yes | installed, removed |
| cpfs_profile_size | Profile size which affect replicas and resources of Pods of CPFS as per https://www.ibm.com/docs/en/cpfs?topic=operator-hardware-requirements-recommendations-foundational-services | Yes | starterset, small, medium, large |
Used to configure CP4BA.
Placed in cp4ba key on the project level.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| enabled | Set to true to enable CP4BA. Currently always true. | Yes | true |
| profile_size | Profile size which affect replicas and resources of Pods as per https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=pcmppd-system-requirements | Yes | small, medium, large |
| patterns | Section where CP4BA patterns are configured. Please make sure to select all that is needed as a dependencies. Dependencies can be determined from documentation at https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=deployment-capabilities-production-deployments | Yes | Object - see details below |
Always configure in CP4BA.
https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=deployment-capabilities-production-deployments#concept_c2l_1ks_fnb__foundation
Placed in cp4ba.patterns.foundation key.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| optional_components | Sub object for definition of optional components for pattern. | Yes | Object - specific to each pattern |
| optional_components.bas | Set to true to enable Business Automation Studio | Yes | true, false |
| optional_components.bai | Set to true to enable Business Automation Insights | Yes | true, false |
| optional_components.ae | Set to true to enable Application Engine | Yes | true, false |
Used to configure Operation Decision Manager.
https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=deployment-capabilities-production-deployments#concept_c2l_1ks_fnb__odm
Placed in cp4ba.patterns.decisions key.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| enabled | Set to true to enable decisions pattern. | Yes | true, false |
| optional_components | Sub object for definition of optional components for pattern. | Yes | Object - specific to each pattern |
| optional_components.decision_center | Set to true to enable Decision Center | Yes | true, false |
| optional_components.decision_runner | Set to true to enable Decision Runner | Yes | true, false |
| optional_components.decision_server_runtime | Set to true to enable Decision Server | Yes | true, false |
| cr_custom | Additional customization for Operational Decision Management. Contents will be merged into ODM part of CP4BA CR yaml file. Arrays are overwritten. | No | Object |
Used to configure Automation Decision Services.
https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=deployment-capabilities-production-deployments#concept_c2l_1ks_fnb__ads
Placed in cp4ba.patterns.decisions_ads key.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| enabled | Set to true to enable decisions_ads pattern. | Yes | true, false |
| optional_components | Sub object for definition of optional components for pattern. | Yes | Object - specific to each pattern |
| optional_components.ads_designer | Set to true to enable Designer | Yes | true, false |
| optional_components.ads_runtime | Set to true to enable Runtime | Yes | true, false |
| gen_ai | Sub object for definition of GenAI connection. More on https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/24.0.0?topic=services-configuring-generative-ai-secret | false | Object |
| gen_ai.apiKey | Set to real value of your Watsonx.AI platform | false | Your real value |
| gen_ai.authUrl | Set to real value of your Watsonx.AI platform | false | Your real value |
| gen_ai.mlUrl | Set to real value of your Watsonx.AI platform | false | Your real value |
| gen_ai.projectId | Set to real value of your Watsonx.AI platform | false | Your real value |
Used to configure FileNet Content Manager.
https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=deployment-capabilities-production-deployments#concept_c2l_1ks_fnb__ecm
Placed in cp4ba.patterns.content key.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| enabled | Set to true to enable content pattern. | Yes | true, false |
| optional_components | Sub object for definition of optional components for pattern. | Yes | Object - specific to each pattern |
| optional_components.cmis | Set to true to enable CMIS | Yes | true, false |
| optional_components.css | Set to true to enable Content Search Services | Yes | true, false |
| optional_components.es | Set to true to enable External Share. Currently not functional. | Yes | true, false |
| optional_components.tm | Set to true to enable Task Manager | Yes | true, false |
| optional_components.ier | Set to true to enable IBM Enterprise Records | Yes | true, false |
| optional_components.icc4sap | Set to true to enable IBM Content Collector for SAP. Currently not functional. Always false. | Yes | false |
Used to configure Business Automation Application.
https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=deployment-capabilities-production-deployments#concept_c2l_1ks_fnb__baa
Placed in cp4ba.patterns.application key.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| enabled | Set to true to enable application pattern. | Yes | true, false |
| optional_components | Sub object for definition of optional components for pattern. | Yes | Object - specific to each pattern |
| optional_components.app_designer | Set to true to enable Application Designer | Yes | true, false |
| optional_components.ae_data_persistence | Set to true to enable App Engine data persistence | Yes | true, false |
Used to configure Automation Document Processing.
https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=deployment-capabilities-production-deployments#concept_c2l_1ks_fnb__baa
Placed in cp4ba.patterns.document_processing key.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| enabled | Set to true to enable document_processing pattern. | Yes | true, false |
| optional_components | Sub object for definition of optional components for pattern. | Yes | Object - specific to each pattern |
| optional_components.document_processing_designer | Set to true to enable Designer | Yes | true |
| cr_custom | Additional customization for Automation Document Processing. Contents will be merged into ADP part of CP4BA CR yaml file. Arrays are overwritten. | No | Object |
Used to configure Business Automation Workflow.
https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=deployment-capabilities-production-deployments#concept_c2l_1ks_fnb__baw
Placed in cp4ba.patterns.workflow key.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| enabled | Set to true to enable workflow pattern. | Yes | true, false |
| optional_components | Sub object for definition of optional components for pattern. | Yes | Object - specific to each pattern |
| optional_components.baw_authoring | Set to true to enable Workflow Authoring. Currently always true. | Yes | true |
| optional_components.kafka | Set to true to enable kafka service for workflow authoring. | Yes | true, false |
Used to configure IBM Process Mining.
Placed in pm key on the project level.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| enabled | Set to true to enable process mining. | Yes | true, false |
| cr_custom | Additional customization for Process Mining. Contents will be merged into PM CR yaml file. Arrays are overwritten. | No | Object |
Used to configure IBM Robotic Process Automation.
Placed in rpa key on the project level.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| enabled | Set to true to enable rpa. | Yes | true, false |
| cr_custom | Additional customization for Process Mining. Contents will be merged into RPA CR yaml file. Arrays are overwritten. | No | Object |
Used to configure extra UIs.
The following properties are defined on the project level.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| cloudbeaver_enabled | Set to true to enable CloudBeaver (PostgreSQL, DB2, MSSQL UI). | Yes | true, false |
| roundcube_enabled | Set to true to enable Roundcube. Client for mail. | Yes | true, false |
| cerebro_enabled | Set to true to enable Cerebro. Client for ElasticSearch in CP4BA. | Yes | true, false |
| akhq_enabled | Set to true to enable AKHQ. Client for Kafka in CP4BA. | Yes | true, false |
| mongo_express_enabled | Set to true to enable Mongo Express. Client for MongoDB. | Yes | true, false |
| phpldapadmin_enabled | Set to true to enable phpLDApAdmin. Client for OpenLDAP. | Yes | true, false |
| opensearch_dashboards_enabled | Set to true to enable OpenSearch Dashboards. Client for OpenSearch. | Yes | true, false |
Contains CP4BA version 23.0.2 iFix 3.
RPA and Process Mining are currently not deployed due to discrepancy in Cloud Pak Foundational Services version.
Contains IPM version 1.14.4. Contains RPA version 23.0.15.
This is not an official IBM documentation.
Absolutely no warranties, no support, no responsibility for anything.
Use it on your own risk and always follow the official IBM documentations.
It is always your responsibility to make sure you are license compliant when using this repository to install IBM Cloud Pak for Business Automation.
Please do not hesitate to create an issue here if needed. Your feedback is appreciated.
Not for production use (neither dev nor test or prod environments). Suitable for Demo and PoC environments - but with Production deployment.
!Important - Keep in mind that this deployment contains capabilities (the ones which are not bundled with CP4BA) which are not eligible to run on Worker Nodes covered by CP4BA OCP Restricted licenses. More info on https://www.ibm.com/docs/en/cloud-paks/1.0?topic=clusters-restricted-openshift-entitlement.
Deploying CP4BA is based on official documentation which is located at https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest.
Deployment of other parts is also based on respective official documentations.
What is not included:
When you perform full deployment, as a result you will get full CP4BA platform as seen in the picture. You can also omit some capabilities - this is covered later in this doc.
More details about each section from the picture follows below it.
Contains extra software which makes working with the platform even easier.
CP4BA capabilities are in purple color.
More info for CP4BA capabilities is available in official docs at https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest.
More specifically in overview of patterns at https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/latest?topic=deployment-capabilities-production-deployments.
Pink color is used for CPFS dedicated capabilities.
More info for CPFS dedicated capabilities is available in official docs at https://www.ibm.com/docs/en/cloud-paks/foundational-services/latest.
Magenta color is used for additional capabilities.
More info for Process Mining is available in official docs at https://www.ibm.com/docs/en/process-mining/latest.
More info for RPA is available in official docs at https://www.ibm.com/docs/en/rpa/latest.
Assets are currently not deployed.
Contains services which are reused by Cloud Paks.
More info available in official docs at https://www.ibm.com/docs/en/cpfs.
Contains prerequisites for the whole platform.
With proper sizing of the cluster and provided RWX File and RWO Block Storage Class, CP4BA deployed with Deployer should be working on any OpenShift 4.14 with Worker Nodes which in total have free 60 CPU, 128GB Memory for requests.
For your convenience the following post-deployment setup tasks have been automated:
Endpoints, access info and other useful information is available in Project cloud-pak-deployer in ConfigMap cp4ba-usage in usage.md file after installation. It is best to copy the contents and open it in nice MarkDown editor like VSCode. The ConfigMap name can begin with a different name if you customized main CP4BA project name.
CP4BA
Review and perform post deploy manual steps for CP4BA as specified in Project cloud-pak-deployer in ConfigMap cp4ba-postdeploy in postdeploy.md file. It is best to copy the contents and open it in nice MarkDown editor like VSCode. The ConfigMap name can begin with a different name if you customized main CP4BA project name.
RPA
Review and perform post deploy manual steps for RPA as specified in Project cloud-pak-deployer in ConfigMap cp4ba-rpa-postdeploy in postdeploy.md file. It is best to copy the contents and open it in nice MarkDown editor like VSCode. The ConfigMap name can begin with a different name if you customized main CP4BA project name.
Process Mining
Review and perform post deploy manual steps for IPM as specified in Project cloud-pak-deployer in ConfigMap cp4ba-pm-postdeploy in postdeploy.md file. It is best to copy the contents and open it in nice MarkDown editor like VSCode. The ConfigMap name can begin with a different name if you customized main CP4BA project name.
Cloud Pak for Data can connect to an external identity provider (IdP) for user authentication. This function is delegated to Foundational Services IAM. Additional to user authentication, the IdP's groups can be mapped to Cloud Pak for Data user groups for access control.
zen_role🔗Cloud Pak Deployer can be used to define user-defined roles in Cloud Pak for Data. Roles identify the permissions that a user or user group has on the platform.
zen_role:
+- name: monitor-role
+ description: User-defined role for monitoring the platform
+ state: installed
+ permissions:
+ - monitor_platform
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the Cloud Pak for Data role | Yes | |
| description | Description of the Cloud Pak for Data role | Yes | |
| state | Indicates if the role must be installed or removed same OpenShift cluster | No | installed (default), removed |
| permissions[] | List of permissions to grant to the role | Yes |
To find the permissions that are allows, you can use the following REST API (GET) after authenticating to the platform: https://$CP4D_URL/icp4d-api/v1/permissions.
zen_access_control🔗The zen_access_control object controls the creation of Zen user groups that map identify provider (IdP) groups and define the roles of the user group. A user_groups entry must contain at least 1 roles and must reference the associated IdP grouop(s).
Example with Red Hat SSO (Keycloak) authentication
zen_access_control:
+- project: cpd
+ openshift_cluster_name: "{{ env_id }}"
+ keycloak_name: ibm-keycloak
+ user_groups:
+ - name: cp4d-admins
+ description: Cloud Pak for Data Administrators
+ roles:
+ - Administrator
+ keycloak_groups:
+ - kc-cp4d-admins
+ - name: cp4d-data-engineers
+ description: Cloud Pak for Data Data Engineers
+ roles:
+ - User
+ keycloak_groups:
+ - kc-cp4d-data-engineers
+ - name: cp4d-data-scientists
+ description: Cloud Pak for Data Data Scientists
+ roles:
+ - User
+ keycloak_groups:
+ - kc-cp4d-data-scientists
+ - name: cp4d-monitor
+ description: Cloud Pak for Data monitoring
+ roles:
+ - monitor-role
+ keycloak_groups:
+ - kc-cp4d-monitor
+Example with OpenLDAP authentication
zen_access_control:
+- project: cpd
+ openshift_cluster_name: "{{ env_id }}"
+ demo_openldap_name: ibm-openldap
+ user_groups:
+ - name: cp4d-admins
+ description: Cloud Pak for Data Administrators
+ roles:
+ - Administrator
+ ldap_groups:
+ - cn=cp4d-admins,ou=Groups,dc=cp,dc=internal
+ - name: cp4d-data-engineers
+ description: Cloud Pak for Data Data Engineers
+ roles:
+ - User
+ ldap_groups:
+ - cn=cp4d-data-engineers,ou=Groups,dc=cp,dc=internal
+ - name: cp4d-data-scientists
+ description: Cloud Pak for Data Data Scientists
+ roles:
+ - User
+ ldap_groups:
+ - cn=cp4d-data-scientists,ou=Groups,dc=cp,dc=internal
+ - name: cp4d-monitor
+ description: Cloud Pak for Data monitoring
+ roles:
+ - monitor-role
+ ldap_groups:
+ - cn=cp4d-monitor,ou=Groups,dc=cp,dc=internal
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| project | project of the cp4d instance | Yes | |
| openshift_cluster_name | Reference to the openshift name | Yes | |
| keycloak_name | Name of the Red Hat SSO (Keycloak) instance on the same OpenShift cluster | No | |
| demo_openldap_name | Name of the OpenLDAP instance defined in the demo_openldap resoureester | No | |
| user_groups[] | Cloud Pak for Data user groups to be configured | Yes | |
| .name | Name of the CP4D user group | Yes | |
| .description | Description of the CP4D user group | No | |
| .roles[] | List of CP4D roles to assign to the user grouop | Yes | |
| .keycloak_groups[] | List of Red Hat SSO (Keycloak) groups to assign to the CP4D user group | Yes if IdP is Keycloak | |
| .ldap_groups[] | List of OpenLDAP groups to assign to the CP4D user group | Yes if IdP is OpenLDAP |
role values: The following roles are defined by default in Cloud Pak for Data: - Administrator - User
Further roles can be defined in the zen object and can be referenced by the user_groups.roles[] property.
During the creation of User Group(s) the following validations are performed: - The provided role(s) are available in Cloud Pak for Data
cp4d_instance_configuration🔗When using Cloud Pak for Data LDAP connectivity and User Groups, the User Groups can be assigned to authorize the users of the LDAP groups access to the proviosioned instance(s).
Currently supported instance authorization:
- Cognos Analytics (ca)
cp4d_instance_configuration:
+- project: zen-sample # Mandatory
+ openshift_cluster_name: sample # Mandatory
+ cartridges:
+ - name: cognos_analytics
+ manage_access: # Optional, requires LDAP connectivity
+ - ca_role: Analytics Viewer # Mandatory, one the CA Access roles
+ cp4d_user_group: CA_Analytics_Viewer # Mandatory, the CP4D User Group Name
+ - ca_role: Analytics Administrators # Mandatory, one the CA Access roles
+ cp4d_user_group: CA_Analytics_Administrators # Mandatory, the CP4D User Group Name
+A Cognos Analytics (ca) instance can have multiple manage_access entries. Each entry consists of 1 ca_role and 1 cp4d_user_group element. The ca_role must be one of the following possible values: - Analytics Administrators - Analytics Explorers - Analytics Users - Analytics Viewer
During the configuration of the instance authorization the following validations are performend: - LDAP configuration is completed - The provided ca_role is valid - The provided cp4d_user_group exists
IBM Cloud Pak for Data can connect to an LDAP user registry in order for users to log on with their LDAP credentials. The configuration of LDAP can be specified in a seperate yaml file in the config folder, or included in an existing yaml file.
cp4d_ldap_config🔗A cp4d_ldap_config entry contains the connectivity information to the LDAP user registry. The project and openshift_cluster_name values uniquely identify the Cloud Pak for Data instance. The ldap_domain_search_password_vault entry contains a reference to the vault, which means that as a preparation the LDAP bind user password must be stored in the vault used by the Cloud Pak Deployer using the key referenced in the configuration. If the password is not available, the Cloud Pak Deployer will fail and not able to configure the LDAP connectivity.
# Each Cloud Pak for Data Deployment deployed in an OpenShift Project of an OpenShift cluster can have its own LDAP configuration
+cp4d_ldap_config:
+- project: cpd-instance
+ openshift_cluster_name: sample # Mandatory
+ ldap_host: ldaps://ldap-host # Mandatory
+ ldap_port: 636 # Mandatory
+ ldap_user_search_base: ou=users,dc=ibm,dc=com # Mandatory
+ ldap_user_search_field: uid # Mandatory
+ ldap_domain_search_user: uid=ibm_roks_bind_user,ou=users,dc=ibm,dc=com # Mandatory
+ ldap_domain_search_password_vault: ldap_bind_password # Mandatory, Password vault reference
+ auto_signup: "false" # Mandatory
+ ldap_group_search_base: ou=groups,dc=ibm,dc=com # Optional, but mandatory when using user groups
+ ldap_group_search_field: cn # Optional, but mandatory when using user groups
+ ldap_mapping_first_name: cn # Optional, but mandatory when using user groups
+ ldap_mapping_last_name: sn # Optional, but mandatory when using user groups
+ ldap_mapping_email: mail # Optional, but mandatory when using user groups
+ ldap_mapping_group_membership: memberOf # Optional, but mandatory when using user groups
+ ldap_mapping_group_member: member # Optional, but mandatory when using user groups
+The above configuration uses the LDAPS protocol to connect to port 636 on the ldap-host server. This server can be a private server if an upstream DNS server is also defined for the OpenShift cluster that runs Cloud Pak for Data. Common Name uid=ibm_roks_bind_user,ou=users,dc=ibm,dc=com is used as the bind user for the LDAP server and its password is retrieved from vault secret ldap_bind_password.
The Cloud Pak Deployer can implement demo assets and accelerators as part of the deployment process to standardize standing up fully-featured demo environments, or to test patches or new versions of the Cloud Pak using pre-defined assets.
If you put a script named apply-custom-node-settings.sh in the CONFIG_DIR/assets directory, it will be run as part of applying the node settings. This way you can override the existing node settings applied by the deployer or update the compute nodes with new settings. For more information regarding the apply-custom-node-settings.sh script, go to Prepare OpenShift cluster on IBM Cloud and IBM Cloud Satellite.
cp4d_asset🔗A cp4d_asset entry defines one or more assets to be deployed for a specific Cloud Pak for Data instance (OpenShift project). In the configuration, a directory relative to the configuration directory (CONFIG_DIR) is specified. For example, if the directory where the configuration is stored is $HOME/cpd-config/sample and you specify assets as the asset directory, all assets under /cpd-config/sample/assets are processed.
You can create one or more subdirectories under the specified location, each holding an asset to be deployed. The deployer finds all cp4d-asset.sh scripts and cp4d-asset.yaml Ansible task files and runs them.
The following runtime attributes will be set prior to running the shell script or the Ansible task: * If the Cloud Pak for Data instances has the Common Core Services (CCS) custom resource installed, cpdctl is configured for the current Cloud Pak for Data instance and the current context is set to the admin user of the instance. This means you can run all cpdctl commands without first having to login to Cloud Pak for Data. * * The current working directory is set to the directory holding the cp4d-asset.sh script. * When running the cp4d-asset.sh shell script, the following environment variables are available: - CP4D_URL: Cloud Pak for Data URL - CP4D_ADMIN_PASSWORD: Cloud Pak for Data admin password - CP4D_OCP_PROJECT: OpenShift project that holds the Cloud Pak for Data instance - KUBECONFIG: OpenShift configuration file that allows you to run oc commands for the cluster
cp4d_asset:
+- name: sample-asset
+ project: cpd
+ asset_location: cp4d-assets
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the asset to be deployed. You can specify as many assets as wanted | Yes | |
| project | Name of OpenShift project of the matching cp4d entry. The cp4d project must exist. | Yes | |
| asset_location | Directory holding the asset(s). This is a directory relative to the config directory (CONFIG_DIR) that was passed to the deployer | Yes |
Below is an example asset that implements the Customer Attrition industry accelerator, which can be found here: https://github.com/IBM/Industry-Accelerators/blob/master/CPD%204.0.1.0/utilities-customer-attrition-prediction-industry-accelerator.tar.gz
To implement:
cp4d-assets directory in the specified configuration directorycp4d-asset.sh shell script (example below)cp4d_asset entry to the Cloud Pak for Data config file in the config directory (or in any other file with extention .yaml)cp4d-asset.sh shell script:
#!/bin/bash
+SCRIPT_DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )
+
+# Function to retrieve project by name
+function retrieve_project {
+ project_name=$1
+
+ # First check if project already exists
+ project_id=$(cpdctl project list \
+ --output json | \
+ jq -r --arg project_name $project_name \
+ 'if .total_results==0 then "" else .resources[] | select(.entity.name == $project_name) | .metadata.guid end')
+
+ echo $project_id
+}
+
+# Function to create a project
+function create_project {
+ project_name=$1
+
+ retrieve_project $project_name
+
+ if [ "$project_id" != "" ];then
+ echo "Project $project_name already exists"
+ return
+ else
+ echo "Creating project $project_name"
+ storage_id=$(uuidgen)
+ storage=$(jq --arg storage_id $storage_id '. | .guid=$storage_id | .type="assetfiles"' <<< '{}')
+ cpdctl project create --name $project_name --storage "$storage"
+ fi
+
+ # Find project_id to return
+ project_id=$(cpdctl project list \
+ --output json | \
+ jq -r --arg project_name $project_name \
+ 'if .total_results==0 then "" else .resources[] | select(.entity.name == $project_name) | .metadata.guid end')
+}
+
+# Function to import a project
+function import_project {
+ project_id=$1
+ zip_file=$2
+ import_id=$(cpdctl asset import start \
+ --project-id $project_id --import-file $zip_file \
+ --output json --jmes-query "metadata.id" --raw-output)
+
+ cpdctl asset import get --project-id $project_id --import-id $import_id --output json
+
+}
+
+# Function to run jobs
+function run_jobs {
+ project_id=$1
+ for job in $(cpdctl job list --project-id $project_id \
+ --output json | jq -r '.results[] | .metadata.asset_id');do
+ cpdctl job run create --project-id $project_id --job-id $job --job-run "{}"
+ done
+}
+
+#
+# Start of the asset code
+#
+
+# Unpack the utilities-customer-attrition-prediction-industry-accelerator directory
+rm -rf /tmp/utilities-customer-attrition-prediction-industry-accelerator
+tar xzf utilities-customer-attrition-prediction-industry-accelerator.tar.gz -C /tmp
+asset_dir=/tmp/customer-attrition-prediction-industry-accelerator
+
+# Change to the asset directory
+pushd ${asset_dir} > /dev/null
+
+# Log on to Cloud Pak for Data with the admin user
+cp4d_token=$(curl -s -k -H 'Content-Type: application/json' -X POST $CP4D_URL/icp4d-api/v1/authorize -d '{"username": "admin", "password": "'$CP4D_ADMIN_PASSWORD'"}' | jq -r .token)
+
+# Import categories
+curl -s -k -H 'accept: application/json' -H "Authorization: Bearer ${cp4d_token}" -H "content-type: multipart/form-data" -X POST $CP4D_URL/v3/governance_artifact_types/category/import?merge_option=all -F "file=@./utilities-customer-attrition-prediction-glossary-categories.csv;type=text/csv"
+
+# Import glossary terms
+curl -s -k -H 'accept: application/json' -H "Authorization: Bearer ${cp4d_token}" -H "content-type: multipart/form-data" -X POST $CP4D_URL/v3/governance_artifact_types/glossary_term/import?merge_option=all -F "file=@./utilities-customer-attrition-prediction-glossary-terms.csv;type=text/csv"
+
+# Check if customer-attrition project already exists. If so, do nothing
+project_id=$(retrieve_project "customer-attrition")
+
+# If project does not exist, import it and run jobs
+if [ "$project_id" == "" ];then
+ create_project "customer-attrition"
+ import_project $project_id \
+ /tmp/utilities-customer-attrition-prediction-industry-accelerator/utilities-customer-attrition-prediction-analytics-project.zip
+ run_jobs $project_id
+else
+ echo "Skipping deployment of CP4D asset, project customer-attrition already exists"
+fi
+
+# Return to original directory
+popd > /dev/null
+
+exit 0
+Defines the services (cartridges) which must be installed into the Cloud Pak for Data instances. The cartridges will be configured with the storage class defined at the Cloud Pak for Data object level. For each cartridge you can specify whether it must be installed or removed by specifying the state. If a cartridge is installed and the state is changed to removed, the cartridge and all of its instances are removed by the deployer when it is run.
An example Cloud Pak for Data object with cartridges is below:
cp4d:
+- project: cpd-instance
+ cp4d_version: 4.8.3
+
+ cartridges:
+ - name: cpfs
+
+ - name: cpd_platform
+
+ - name: db2oltp
+ size: small
+ instances:
+ - name: db2-instance
+ metadata_size_gb: 20
+ data_size_gb: 20
+ backup_size_gb: 20
+ transactionlog_size_gb: 20
+ state: installed
+
+ - name: wkc
+ size: small
+ state: removed
+
+ - name: wml
+ size: small
+ state: installed
+
+ - name: ws
+ state: installed
+When run, the deployer installs the Db2 OLTP (db2oltp), Watson Machine Learning (wml) and Watson Studio (ws) cartridges. If the Watson Knowledge Catalog (wkc) is installed in the cpd-instance OpenShift project, it is removed.
After the deployer installs Db2 OLTP, a new Db2 instance is created with the specified attributes.
cp4d.cartridges🔗This is a list of cartridges that will be installed in the Cloud Pak for Data instance. Every cartridge is identified by its name.
Some cartridges may require additional information to correctly install or to create an instance for the cartridge. Below you will find a list of all tested Cloud Pak for Data cartridges and their specific properties.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the cartridge | Yes | |
| state | Whether the cartridge must be installed or removed. If not specified, the cartridge will be installed | No | installed, removed |
| installation_options | Record of properties that will be applied to the spec of the OpenShift Custom Resource | No |
cpfs or cp-foundation🔗Defines the Cloud Pak Foundational Services (fka Common Services) which are required for all Cloud Pak for Data installations. Cloud Pak for Data Foundational Services provide functionalities around certificate management, license service, identity and access management (IAM), etc.
This cartridge is mandatory for every Cloud Pak for Data instance.
cpd_platform or lite🔗Defines the Cloud Pak for Data platform operator (fka "lite") which installs the base services needed to operate Cloud Pak for Data, such as the Zen metastore, Zen watchdog and the user interface.
This cartridge is mandatory for every Cloud Pak for Data instance.
wkc🔗Manages the Watson Knowledge Catalog installation for the Cloud Pak for Data instance.
wkc🔗| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| size | Scale configuration of the cartridge | No | small (default), medium, large |
| installation_options.install_wkc_core_only | Install only the core of WKC? | No | True, False (default) |
| installation_options.enableKnowledgeGraph | Enable the knowledge graph for business lineage? | No | True, False (default) |
| installation_options.enableDataQuality | Enable data quality for WKC? | No | True, False (default) |
| installation_options.enableMANTA | Enable MANTA? | No | True, False (default) |
cp4d_conection🔗The cp4d_connection object can be used to create Global Platform connections.
cp4d_connection:
+- name: connection_name # Name of the connection, must be unique
+ type: database # Type, currently supported: [database]
+ cp4d_instance: cpd # CP4D instance on which the connection must be created
+ openshift_cluster_name: cluster_name # OpenShift cluster name on which the cp4d_instance is deployed
+ database_type: db2 # Type of connection
+ database_hostname: hostname # Hostname of the connection
+ database_port: 30556 # Port of the connection
+ database_name: bludb # Database name of the connection
+ database_port_ssl: true # enable ssl flag
+ database_credentials_username: 77066f69 # Username of the datasource
+ database_credentials_password_secret: db-credentials # Vault lookup name to contain the password
+ database_ssl_certificate_secret: db-ssl-cert # Vault lookup name to contain the SSL certificate
+cp4d_backup_restore_connections🔗The cp4d_backup_restore_connections can be used to backup all current configured Global Platform connections, which are either created by the Cloud Pak Deployer or added manually. The backup is stored in the status/cp4d/exports folder as a json file.
A backup file can be used to restore global platform connections. A flag can be used to indicate whether if a Global Platform connection with the same name already exists, the restore is skipped.
Using the Cloud Pak Deployer cp4d_backup_restore_connections capability implements the following: - Connect to the IBM Cloud Pak for Data instance specified using cp4d_instance and openshift_cluster_name - If connections_backup_file is specified export all Global Platform connections to the specified file in the status/cp4d/export/connections folder - If connections_restore_file is specified, load the file and restore the Global Platform connections - The connections_restore_overwrite (true/false) indicates whether if a Global Platform Connection with the same already exists, it will be replaced.
cp4d_backup_restore_connections:
+- cp4d_instance: cpd
+ openshift_cluster_name: {{ env_id }}
+ connections_backup_file: {{ env_id }}_cpd_connections.json
+ connections_restore_file: {{ env_id }}_cpd_connection.json
+ connections_restore_overwrite: false
+Some cartridges have the ability to create one or more instances to run an isolated installation of the cartridge. If instances have been configured for the cartridge, the deployer can manage creating and deleting the instances.
The following Cloud Pak for Data cartridges are currently supported for managing instances:
analytics-engine)datastage-ent-plus) db2)dv)ca)edb_cp4d)openpages)Analytics Engine instances can be defined by adding the instances section to the cartridges entry of cartridge analytics-engine. The following example shows the configuration to define an instance.
cp4d:
+- project: cpd-instance
+ openshift_cluster_name: "{{ env_id }}"
+...
+ cartridges:
+ - name: analytics-engine
+ size: small
+ state: installed
+ instances:
+ - name: analyticsengine-instance
+ storage_size_gb: 50
+| Property | Description | Mandatory | Allowed Values |
|---|---|---|---|
| name | Name of the instance | Yes | |
| storage_size_db | Size of the storage allocated to the instance | Yes | numeric value |
DataStage instances can be defined by adding the instances section to the cartridges entry of cartridge datastage-ent-plus. The following example shows the configuration to define an instance.
DataStage, upon deployment, always creates a default instance called ds-px-default. This instance cannot be configured in the instances section.
cp4d:
+- project: cpd-instance
+ openshift_cluster_name: "{{ env_id }}"
+...
+ cartridges:
+ - name: datastage-ent-plus
+ state: installed
+
+ instances:
+ - name: ds-instance
+ # Optional settings
+ description: "datastage ds-instance"
+ size: medium
+ storage_class: efs-nfs-client
+ storage_size_gb: 60
+ # Optional Custom Scale options
+ scale_px_runtime:
+ replicas: 2
+ cpu_request: 500m
+ cpu_limit: 2
+ memory_request: 2Gi
+ memory_limit: 4Gi
+ scale_px_compute:
+ replicas: 2
+ cpu_request: 1
+ cpu_limit: 3
+ memory_request: 4Gi
+ memory_limit: 12Gi
+| Property | Description | Mandatory | Allowed Values |
|---|---|---|---|
| name | Name of the instance | Yes | |
| description | Description of the instance | No | |
| size | Size of the DataStage instance | No | small (default), medium, large |
| storage_class | Override the default storage class | No | |
| storage_size_gb | Storage size allocated to the DataStage instance | No | numeric |
Optionally, the default px_runtime and px_compute instances of the DataStage instance can be tweaked. Both scale_px_runtime and scale_px_compute must be specified when used, and all properties must be specified.
| Property | Description | Mandatory |
|---|---|---|
| replicas | Number of replicas | Yes |
| cpu_request | CPU Request value | Yes |
| memory_request | Memory Request value | Yes |
| cpu_limit | CPU limit value | Yes |
| memory_limit | Memory limit value | Yes |
DB2 OLTP instances can be defined by adding the instances section to the cartridges entry of cartridge db2. The following example shows the configuration to define an instance.
cp4d:
+- project: cpd-instance
+ openshift_cluster_name: "{{ env_id }}"
+...
+ cartridges:
+ - name: db2
+ size: small
+ state: installed
+ instances:
+ - name: db2 instance
+ metadata_size_gb: 20
+ data_size_gb: 20
+ backup_size_gb: 20
+ transactionlog_size_gb: 20
+
+| Property | Description | Mandatory | Allowed Values |
|---|---|---|---|
| name | Name of the instance | Yes | |
| metadata_size_gb | Size of the metadata store | Yes | numeric value |
| data_size_gb | Size of the data store | Yes | numeric value |
| backup_size_gb | Size of the backup store | Yes | numeric value |
| transactionlog_size_gb | Size of the transactionlog store | Yes | numeric value |
Data Virtualization instances can be defined by adding the instances section to the cartridges entry of cartridge dv. The following example shows the configuration to define an instance.
cp4d:
+- project: cpd-instance
+ openshift_cluster_name: "{{ env_id }}"
+...
+ cartridges:
+ - name: dv
+ size: small
+ state: installed
+ instances:
+ - name: data-virtualization
+| Property | Description | Mandatory | Allowed Values |
|---|---|---|---|
| name | Name of the instance | Yes |
A Cognos Analytics instance can be defined by adding the instances section to the cartridges entry of cartridge ca. The following example shows the configuration to define an instance.
cp4d:
+- project: cpd-instance
+ openshift_cluster_name: "{{ env_id }}"
+...
+ cartridges:
+ - name: ca
+ size: small
+ state: installed
+ instances:
+ - name: ca-instance
+ metastore_ref: ca-metastore
+| Property | Description | Mandatory |
|---|---|---|
| name | Name of the instance | Yes |
| metastore_ref | Name of the DB2 instance used for the Cognos Repository database | Yes |
The Cognos Content Repository database can use an IBM Cloud Pak for Data DB2 OLTP instance. The Cloud Pak Deployer will first determine whether an existing DB2 OLTP existing with the name specified metastore_ref. If this is the case, this DB2 OLTP instance will be used and the database is prepared using the Cognos DB2 script prior to provisioning the Cognos instance.
EnterpriseDB instances can be defined by adding the instances section to the cartridges entry of cartridge dv. The following example shows the configuration to define an instance.
cp4d:
+- project: cpd-instance
+ openshift_cluster_name: "{{ env_id }}"
+...
+ cartridges:
+
+ # Please note that for EDB Postgress, a secret edb-postgres-license-key must be created in the vault
+ # before deploying
+ - name: edb_cp4d
+ size: small
+ state: installed
+ instances:
+ - name: instance1
+ version: "13.5"
+ #Optional Parameters
+ type: Standard
+ members: 1
+ size_gb: 50
+ resource_request_cpu: 1000m
+ resource_request_memory: 4Gi
+ resource_limit_cpu: 1000m
+ resource_limit_memory: 4Gi
+| Property | Description | Mandatory | Allowed Values |
|---|---|---|---|
| name | Name of the instance | Yes | |
| version | Version of the EDB PostGres instance | Yes | 12.11, 13.5 |
| type | Enterprise or Standard version | No | Standard (default), Enterprise |
| members | Number of members of the instance | No | number, 1 (default) |
| size_gb | Storage Size allocated to the instance | No | number, 50 (default) |
| resource_request_cpu | Request CPU of the instance | No | 1000m (default) |
| resource_request_memory | Request Memory of the instance | No | 4Gi (default) |
| resource_limit_cpu | Limit CPU of the instance | No | 1000m (default) |
| resource_limit_memory | Limit Memory of the instance | No | 4Gi (default) |
An OpenPages instance can be defined by adding the instances section to the cartridges entry of cartridge openpages. The following example shows the configuration to define an instance.
cp4d:
+- project: cpd-instance
+ openshift_cluster_name: "{{ env_id }}"
+...
+ cartridges:
+ - name: openpages
+ state: installed
+ instances:
+ - name: openpages-instance
+ size: xsmall
+| Property | Description | Mandatory |
|---|---|---|
| name | Name of the instance | Yes |
| size | The size of the OpenPages instances, default is xsmall | No |
You can configure Single Sign-on (SSO) by specifying a SAML server for the Cloud Pak for Data instance, which will take care of authenticating users. SAML configuration can be used in combination with the Cloud Pak for Data LDAP configuration, in which case LDAP complements the identity with access management (groups) for users.
cp4d_saml_config🔗An cp4d_saml_config entry holds connection information, certificates and field configuration that is needed in the exchange between Cloud Pak for Data user management and the identity provider (idP). The entry must created for every Cloud Pak for Data project that requires SAML authentication.
When a cp4d_saml_config entry exists for a certain cp4d project, the user management pods are updated with a samlConfig.json file and then restarted. If an entry is removed later, the file is removed and the pods restarted again. When no changes are needed, the file in the pod is left untouched and no restart takes place.
For more information regarding the Cloud Pak for Data SAML configuration, check the single sign-on documentation: https://www.ibm.com/docs/en/cloud-paks/cp-data/4.0?topic=client-configuring-sso
cp4d_saml_config:
+- project: cpd
+ entrypoint: "https://prepiam.ice.ibmcloud.com/saml/sps/saml20ip/saml20/login"
+ field_to_authenticate: email
+ sp_cert_secret: {{ env_id }}-cpd-sp-cert
+ idp_cert_secret: {{ env_id }}-cpd-idp-cert
+ issuer: "cp4d"
+ identifier_format: ""
+ callback_url: ""
+The above configuration uses the IBM preproduction IAM server to delegate authentication to and authentication is done via the user's e-mail address. An issuer must be configured in the identity provider (idP) and the idP's certificate must be kept in the vault so Cloud Pak for Data can confirm its identity.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| project | Name of OpenShift project of the matching cp4d entry. The cp4d project must exist. | Yes | |
| entrypoint | URL of the identity provider (idP) login page | Yes | |
| field_to_authenticate | Name of the parameter to authenticate with the idP | Yes | |
| sp_cert_secret | Vault secret that holds the private certificate to authenticate to the idP. If not specified, requests will not be signed. | No | |
| idp_cert_secret | Vault secret that holds the public certificate of the idP. This confirms the identity of the idP | Yes | |
| issuer | The name you chose to register the Cloud Pak for Data instance with your idP | Yes | |
| identifier_format | Format of the requests from Cloud Pak for Data to the idP. If not specified, urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress is used | No | |
| callback_url | Specify the callback URL if you want to override the default of cp4d_url/auth/login/sso/callback | No |
The callbackUrl field in the samlConfig.json file is automatically populated by the deployer if it is not specified by the cp4d_saml_config entry. It then consists of the Cloud Pak for Data base URL appended with /auth/login/sso/callback.
Before running the deployer with SAML configuration, ensure that the secret configured for idp_cert_secret exists in the vault. Check Vault configuration for instructions on adding secrets to the vault.
Cloud Pak Deployer can use properties set in the global configuration (global_config) during the deployment process and also as substitution variables in the configuration, such as {{ env_id}} and {{ ibm_cloud_region }}.
The following global_config variables are automatically copied into a "simple" form so they can be referenced in the configuration file(s) and also overridden using the command line.
| Variable name | Description |
|---|---|
environment_name | Name used to group secrets, typically you will specify sample |
cloud_platform | Cloud platform applicable to configuration, such as ibm-cloud, aws, azure |
env_id | Environment ID used in various other configuration objects |
ibm_cloud_region | When Cloud Platform is ibm-cloud, the region into which the ROKS cluster is deployed |
aws_region | When Cloud Platform is aws, the region into which the ROSA/self-managed OpenShift cluster is deployed |
azure_location | When Cloud Platform is azure, the region into which the ARO OpenShift cluster is deployed |
universal_admin_user | User name to be used for admin user (currently not used) |
universal_password | Password to be used for all (admin) users it not specified in the vault |
confirm_destroy | Is destroying of clusters, services/cartridges and instances allowed? |
For all other variables, you can refer to the qualified form, for example: "{{ global_config.division }}"
Sample global configuration:
global_config:
+ environment_name: sample
+ cloud_platform: ibm-cloud
+ env_id: pluto-01
+ ibm_cloud_region: eu-de
+ universal_password: very_secure_Passw0rd$
+ confirm_destroy: False
+If you run the cp-deploy.sh command and specify -e env_id=jupiter-03, this will override the value in the global_config object. The same applies to the other variables.
All objects used by the Cloud Pak Deployer are defined in a yaml format in files in the config directory. You can create a single yaml file holding all objects, or group objects in individual yaml files. At deployment time, all yaml files in the config directory are merged.
To make it easier to navigate the different object types, they have been groups in different tabs. You can also use the index below to find the definitions.
You can install an OpenLDAP service on your OpenShift cluster for demonstration and testing purposes. This way you can experiment with LDAP identity providers in Foundational Services if you don't (yet) have access to an enterprise-ready LDAP service in the organization's infrastructure services.
Note Installing an OpenLDAP server must only be done if you have unrestricted OpenShift Container Platform entitlements. When using the Cloud Pak entitlements for Red Hat OpenShift, installing third-party applications like Bitnami OpenLDAP is not allowed.
demo_openldap🔗A demo_ldap resource in the configuration indicates that the Bitname OpenLDAP service is installed on the specified OpenShift cluster. The default OpenShift poject for the OpenLDAP service is openldap. You can install several instances on the same OpenShift cluster if necessary, each with its own name and openldap_project project.
Sample configuration
demo_openldap:
+- name: cp4d-openldap
+ openshift_cluster_name: "{{ env_id }}"
+ openldap_project: openldap
+ ldap_config:
+ ldap_tls: True
+ bind_admin_user: cn=admin,dc=cp,dc=internal
+ base_dn: dc=cp,dc=internal
+ base_dc: cp
+ base_domain: cp.internal
+ user_ou: Users
+ user_id_attribute: uid
+ user_display_name_attribute: cn
+ user_base_dn: ou=Users,dc=cp,dc=internal
+ user_object_class: inetOrgPerson
+ group_ou: Groups
+ group_id_attribute: cn
+ group_display_name_attribute: cn
+ group_base_dn: ou=Groups,dc=cp,dc=internal
+ group_object_class: groupOfUniqueNames
+ group_member_attribute: uniqueMember
+ users:
+ - uid: ttoussaint
+ givenName: Tara
+ sn: Toussaint
+ mail: ttoussaint@cp.internal
+ - uid: rramones
+ givenName: Rosa
+ sn: Ramones
+ mail: rramones@cp.internal
+ # password: specific_password_for_the_user
+ - uid: ssharpe
+ givenName: Shelly
+ sn: Sharpe
+ mail: ssharpe@cp.internal
+ # password: specific_password_for_the_user
+ - uid: pprimo
+ givenName: Paco
+ sn: Primo
+ mail: pprimo@cp.internal
+ # password: specific_password_for_the_user
+ - uid: rroller
+ givenName: Rico
+ sn: Roller
+ mail: rroller@cp.internal
+ # password: specific_password_for_the_user
+ groups:
+ - cn: cp4d-admins
+ members:
+ - uid=ttoussaint,ou=Users,dc=cp,dc=internal
+ - cn: cp4d-data-engineers
+ members:
+ - uid=rramones,ou=Users,dc=cp,dc=internal
+ - uid=ssharpe,ou=Users,dc=cp,dc=internal
+ - cn: cp4d-data-scientists
+ members:
+ - uid=pprimo,ou=Users,dc=cp,dc=internal
+ - uid=ssharpe,ou=Users,dc=cp,dc=internal
+ - uid=rroller,ou=Users,dc=cp,dc=internal
+ state: installed
+The above configuration installs the OpenLDAP service in OpenShift project openldap and configures it for domain cp.internal. Subsequently, an LDIF file with the Organization Units, Groups and Users is generated and then the OpenLDAP service is started.
The OpenLDAP name is referenced in the Cloud Pak for Data Access Control resource and this is also where the mapping from LDAP groups to Cloud Pak for Data groups takes place.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the OpenLDAP server, for reference by zen_access_control | Yes | |
| openshift_cluster_name | Name of OpenShift cluster into which the OpenLDAP service is installed | Yes. if more than 1 openshift resource in the configuration | |
| openldap_project | OpenShift project into which the OpenLDAP server is installed | No, default is openldap | |
| ldap_config | LDAP configuration | Yes | |
| .ldap_tls | Set to True if the LDAPS protocol just be used to communicate with the LDAP server | No | False (default), True |
| .bind_admin_user | Distinguished name of the user to bind (login) to the LDAP server | Yes | |
| .base_dn | Base domain name, specify through dc components | Yes | |
| .base_dc | First dc component in the base_dn | Yes | |
| .base_domain | Base domain of the LDAP root, specified as cp.internal | Yes | |
| .user_ou | Organizational Unit of users, typically Users | Yes | |
| .user_id_attribute | Attribute used to identify user, typically uid | Yes | |
| .user_display_name_attribute | Common name of the user, typically cn | Yes | |
| .user_base_dn | Base domain name of users, typically user_ou, followed by base_dn | Yes | |
| .user_object_class | Object class of the users, typically inetOrgPerson | Yes | |
| .group_ou | Organizational Unit of groups, typically Groups | Yes | |
| .group_id_attribute | Attribte used to idenfity group, typically cn | Yes | |
| .group_display_name_attribute | Common name of the group, typically cn | Yes | |
| .group_base_dn | Base domain name of groups, typically group_ou, followed by base_dn | Yes | |
| .group_object_class | Object class of the gruops, typically groupOfUniqueNames | Yes | |
| .group_member_attribute | Attribute used for a member (user) of a group, typically uniqueMember | Yes | |
| users[] | List of users to be added to the LDAP configuration | Yes | |
| .uid | User identifier that is used to login to the platform | Yes | |
| .givenName | First name of the user | Yes | |
| .sn | Surname of the user | Yes | |
| e-mail address of the user | Yes | ||
| .password | Password to be assigned to the user. If not specified, the universal password is used | No | |
| groups[] | List of groups to be added to the LDAP configuration | Yes | |
| .cn | Group identifier, together with the group_ou and base_dn, this will define the group to map to the Cloud Pak group(s) | Yes | |
| .members[] | List of user distinguished names to be added as members to the group | Yes | |
| state | Indicates whether or nog OpenLDAP must be installed | Yes | installed, removed |
When deploying OpenShift in a private network, one may want to reach additional private network services by their host name. Examples could be a database server, Hadoop cluster or an LDAP server. OpenShift provides a DNS operator which deploys and manages CoreDNS which takes care of name resolution for pods running inside the container platform, also known as DNS forwarding.
If the services that need to be reachable our registered on public DNS servers, you typically do not have to configure upstream DNS servers.
The upstream DNS used for a particular OpenShift cluster is configured like this:
openshift:
+- name: sample
+...
+ upstream_dns:
+ - name: sample-dns
+ zones:
+ - example.com
+ dns_servers:
+ - 172.31.2.73:53
+The zones which have been defined for each of the upstream_dns configurations control which DNS server(s) will be used for name resolution. For example, if example.com is given as the zone and an upstream DNS server of 172.31.2.73:53, any host name matching *.example.com will be resolved using DNS server 172.31.2.73 and port 53.
If you want to remove the upstream DNS that was previously configured, you can change the deployer configuration as below and run the deployer. Removing the upstream_dns element altogether will not make changes to the OpenShift DNS operator.
upstream_dns: []
+See https://docs.openshift.com/container-platform/4.8/networking/dns-operator.html for more information about the operator that is configured by specifying upstream DNS servers.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| upstream_dns[] | List of alternative upstream DNS servers(s) for OpenShift | No | |
| name | Name of the upstream DNS entry | Yes | |
| zones | Specification of one or more zone for which the DNS server is applicable | Yes | |
| dns_servers | One or more DNS servers (host:port) that will resolve host names in the specified zone | Yes |
For some of the cloud platforms, you must explicitly specify the infrastructure layer on which the OpenShift cluster(s) will be provisioned, or you can override the defaults.
For IBM Cloud, you can configure the VPC, subnets, NFS server(s), other Virtual Server Instance(s) and a number of other objects. When provisioning OpenShift on vSphere, you can configure data center, data store, network and virtual machine definitions. For Azure ARO you configure a single object with information about the virtual network (vnet) to be used and the node server profiles. When deploying OpenShift on AWS you can specify an EFS server if you want to use elastic storage.
This page lists all the objects you can configure for each of the supported cloud providers. - IBM Cloud - Microsoft Azure - Amazon AWS - vSphere
For IBM Cloud, the following object types are supported:
provider🔗Defines the provider that Terraform will use for managing the IBM Cloud assets.
provider:
+- name: ibm
+ region: eu-de
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the provider cluster | No | ibm |
| region | Region to connect to | Yes | Any IBM Cloud region |
resource_group🔗The resource group is for cloud asset grouping purposes. You can define multiple resource groups in your IBM cloud account to group the provisioned assets. If you do not need to group your assets, choose default.
resource_group:
+- name: default
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the existing resource group | Yes |
ssh_keys🔗SSH keys to connect to VSIs. If you have Virtual Server Instances in your VPC, you will need an SSH key to connect to them. SSH keys defined here will be looked up in the vault and created if they don't exist already.
ssh_keys:
+- name: vsi-access
+ managed: True
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the SSH key in IBM Cloud | Yes | |
| managed | Determines if the SSH key will be created if it doesn't exist | No | True (default), False |
security_rule🔗Defines the services (or ports) which are allowed within the context of a VPC and/or VSI.
security_rule:
+- name: https
+ tcp: {port_min: 443, port_max: 443}
+- name: ssh
+ tcp: {port_min: 22, port_max: 22}
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the security rule | Yes | |
| tcp | Range of tcp ports (port_min and port_max) to allow | No | 1-65535 |
| udp | Range of udp ports (port_min and port_max) to allow | No | 1-65535 |
| icmp | ICMP Type and Code for IPv4 (code and type) to allow | No | 1-255 for code, 1-254 for type |
vpc🔗Defines the virtual private cloud which groups the provisioned objects (including VSIs and OpenShift cluster).
vpc:
+- name: sample
+ allow_inbound: ['ssh', 'https']
+ classic_access: false
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the Virtual Private Cloud | Yes | |
| managed | Controls whether the VPC is managed. The default is True. Only set to False if the VPC is not managed but only referenced by other objects such as transit gateways. | No | True (default), False |
| allow_inbound | Security rules which are allowed for inbound traffic | No | Existing security_rule |
| classic_access | Connect VPC to IBM Cloud classic infratructure resources | No | false (default), true |
address_prefix🔗Defines the zones used within the VPC, along with the subnet the addresses will be issued for.
- name: sample-zone-1
+ vpc: sample
+ zone: eu-de-1
+ cidr: 10.27.0.0/26
+- name: sample-zone-2
+ vpc: sample
+ zone: eu-de-2
+ cidr: 10.27.0.64/26
+- name: sample-zone-3
+ vpc: sample
+ zone: eu-de-3
+ cidr: 10.27.0.128/26
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the zone | Yes | |
| zone | Zone in the IBM Cloud | Yes | |
| cidr | Address range that IPs in this zone will fall into | Yes | |
| vpc | Virtual Private Cloud this address prefix belongs to | Yes, inferred from vpc | Existing vpc |
subnet🔗Defines the subnet that Virtual Server Instances and ROKS compute nodes will be attached to.
subnet:
+- name: sample-subnet-zone-1
+ address_prefix: sample-zone-1
+ ipv4_cidr_block: 10.27.0.0/26
+ zone: eu-de-1
+ vpc: sample
+ network_acl: sample-acl
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the subnet | Yes | |
| zone | Zone this subnet belongs to | Yes, inferred from address_prefix->zone | |
| ipv4_cidr_block | Address range that IPs in this subnet will fall into | Yes, inferred from address_prefix->cidr | Range of subrange of zone |
| address_prefix | Zone of the address prefix definition | Yes, inferred from address_prefix | Existing address_prefix |
| vpc | Virtual Private Cloud this subnet prefix belongs to | Yes, inferred from address_prefix->vpc | Existing vpc |
| network_acl | Reference to the network access control list protecting this subnet | No |
network_acl🔗Defines the network access control list to be associated with subnets to allow or deny traffic from or to external connections. The rules are processed in sequence per direction. Rules that appear higher in the list will be processed first.
network_acl:
+- name: "{{ env_id }}-acl"
+ vpc_name: "{{ env_id }}"
+ rules:
+ - name: inbound-ssh
+ action: allow # Can be allow or deny
+ source: "0.0.0.0/0"
+ destination: "0.0.0.0/0"
+ direction: inbound
+ tcp:
+ source_port_min: 1 # optional
+ source_port_max: 65535 # optional
+ dest_port_min: 22 # optional
+ dest_port_max: 22 # optional
+ - name: output-udp
+ action: deny # Can be allow or deny
+ source: "0.0.0.0/0"
+ destination: "0.0.0.0/0"
+ direction: outbound
+ udp:
+ source_port_min: 1 # optional
+ source_port_max: 65535 # optional
+ dest_port_min: 1000 # optional
+ dest_port_max: 2000 # optional
+ - name: output-icmp
+ action: allow # Can be allow or deny
+ source: "0.0.0.0/0"
+ destination: "0.0.0.0/0"
+ direction: outbound
+ icmp:
+ code: 1
+ type: 1
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the network access control liet | Yes | |
| vpc_name | Virtual Private Cloud this network ACL belongs to | Yes | |
| rules | Rules to be applied, every rule is an entry in the list | Yes | |
| rules.name | Unique name of the rule | Yes | |
| rules.action | Defines whether the traffic is allowed or denied | Yes | allow, deny |
| rules.source | Source address range that defines the rule | Yes | |
| rules.destination | Destination address range that defines the rule | Yes | |
| rules.direction | Inbound or outbound direction of the traffic | Yes | inbound, outbound |
| rules.tcp | Rule for TCP traffic | No | |
| rules.tcp.source_port_min | Low value of the source port range | No, default=1 | 1-65535 |
| rules.tcp.source_port_max | High value of the source port range | No, default=65535 | 1-65535 |
| rules.tcp.dest_port_min | Low value of the destination port range | No, default=1 | 1-65535 |
| rules.tcp.dest_port_max | High value of the destination port range | No, default=65535 | 1-65535 |
| rules.udp | Rule for UDP traffic | No | |
| rules.udp.source_port_min | Low value of the source port range | No, default=1 | 1-65535 |
| rules.udp.source_port_max | High value of the source port range | No, default=65535 | 1-65535 |
| rules.udp.dest_port_min | Low value of the destination port range | No, default=1 | 1-65535 |
| rules.udp.dest_port_max | High value of the destination port range | No, default=65535 | 1-65535 |
| rules.icmp | Rule for ICMP traffic | No | |
| rules.icmp.code | ICMP traffic code | No, default=all | 0-255 |
| rules.icmp.type | ICMP traffic type | No, default=all | 0-254 |
vsi🔗Defines a Virtual Server Instance within the VPC.
vsi:
+- name: sample-bastion
+ infrastructure:
+ type: vpc
+ keys:
+ - "vsi-access"
+ image: ibm-redhat-8-3-minimal-amd64-3
+ subnet: sample-subnet-zone-1
+ primary_ipv4_address: 10.27.0.4
+ public_ip: True
+ vpc_name: sample
+ zone: eu-de-3
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the Virtual Server Instance | Yes | |
| infrastructure | Infrastructure attributes | Yes | |
| infrastructure.type | Infrastructure type | Yes | vpc |
| infrastructure.allow_ip_spoofing | Decide if IP spoofing is allowed for the interface or not | No | False (default), True |
| infrastructure.keys | List of SSH keys to attach to the VSI | Yes, inferred from ssh_keys | Existing ssh_keys |
| infrastructure.image | Operating system image to be used | Yes | Existing image in IBM Cloud |
| infrastructure.profile | Server profile to be used, for example cx2-2x4 | Yes | Existing profile in IBM Cloud |
| infrastructure.subnet | Subnet the VSI will be connected to | Yes, inferred from sunset | Existing subnet |
| infrastructure.primary_ipv4_address | IP v4 address that will be assigned to the VSI | No | If specified, address in the subnet range |
| infrastructure.public_ip | Must a public IP address be attached to this VSI? | No | False (default), True |
| infrastructure.vpc_name | Virtual Private Cloud this VSI belongs to | Yes, inferred from vpc | Existing vpc |
| infrastructure.zone | Zone the VSI will be plaed into | Yes, inferred from subnet->zone |
transit_gateway🔗Connects one or more VPCs to each other.
transit_gateway:
+- name: sample-tgw
+ location: eu-de
+ connections:
+ - vpc: other-vpc
+ - vpc: sample
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the transit gateway | Yes | |
| location | IBM Cloud location of the transit gateway | Yes | |
| connections | Defines which VPCs must be included in the transit gateway | Yes | |
| connection.vpc | Defines the VPC to include. Every VPC must exist in the configuration, even if not managed by this configuration. When referencing an existing VPC, make sure that there is a vpc object of that name with managed set to False. | Yes | Existing vpc |
nfs_server🔗Defines a Virtual Server Instance within the VPC that will be used as an NFS server.
nfs_server:
+- name: sample-nfs
+ infrastructure:
+ type: vpc
+ vpc_name: sample
+ subnet: sample-subnet-zone-1
+ zone: eu-de-1
+ primary_ipv4_address: 10.27.0.5
+ image: ibm-redhat-8-3-minimal-amd64-3
+ profile: cx2-2x4
+ bastion_host: sample-bastion
+ storage_folder: /data/nfs
+ storage_profile: 10iops-tier
+ keys:
+ - "sample-nfs-provision"
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the Virtual Server Instance | Yes | |
| infrastructure | Infrastructure attributes | Yes | |
| infrastructure.image | Operating system image to be used | Yes | Existing image in IBM Cloud |
| infrastructure.profile | Server profile to be used, for example cx2-2x4 | Yes | Existing profile in IBM Cloud |
| infrastructure.type | Type of infrastructure for NFS servers to | Yes | vpc |
| infrastructure.vpc_name | Virtual Private Cloud this VSI belongs to | Yes, inferred from vpc | Existing vpc |
| infrastructure.subnet | Subnet the VSI will be connected to | Yes, inferred from subnet | Existing subnet |
| infrastructure.zone | Zone the VSI will be plaed into | Yes, inferred from subnet->zone | |
| infrastructure.primary_ipv4_address | IP v4 address that will be assigned to the VSI | No | If specified, address in the subnet range |
| infrastructure.bastion_host | Specify the VSI of the bastion to reach this NFS server | No | |
| infrastructure.storage_profile | Storage profile that will be used | Yes | 3iops-tier, 5iops-tier, 10iops-tier |
| infrastructure.volume_size_gb | Size of the NFS server data volume | Yes | |
| infrastructure.storage_folder | Folder that holds the data, this will be mounted from the NFS storage class | Yes | |
| infrastructure.keys | List of SSH keys to attach to the NFS server VSI | Yes, inferred from ssh_keys | Existing ssh_keys |
| infrastructure.allow_ip_spoofing | Decide if IP spoofing is allowed for the interface or not | No | False (default), True |
cos🔗Defines a IBM Cloud Cloud Object Storage instance and allows to create buckets.
cos:
+- name: {{ env_id }}-cos
+ plan: standard
+ location: global
+ serviceids:
+ - name: {{ env_id }}-cos-serviceid
+ roles: ["Manager", "Viewer", "Administrator"]
+ buckets:
+ - name: bucketone6c9d6840
+ cross_region_location: eu
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the serviceid | Yes | |
| plan | short description of the serviceid | Yes | |
| location | collection of servicekeys that should be created for the parent serviceid | Yes | |
| serviceids | Collection of references to defined seriveids | No | |
| serviceids.name | Name of the serviceid | Yes | |
| serviceids.roles | An array of strings to define which role should be granted to the serviceid | Yes | |
| buckets | Collection of buckets that should be created inside the cos instance | No | |
| buckets[].name | Name of the bucket | No | |
| buckets[].storage_class | Storage class of the bucket | No | standard (default), vault, cold, flex, smart |
| buckets[].endpoint_type | Endpoint type of the bucket | No | public (default), private |
| buckets[].cross_region_location | If you use this parameter, do not set single_site_location or region_location at the same time. | Yes (one of) | us, eu, ap |
| buckets[].region_location | If you set this parameter, do not set single_site_location or cross_region_location at the same time. | Yes (one of) | au-syd, eu-de, eu-gb, jp-tok, us-east, us-south, ca-tor, jp-osa, br-sao |
| buckets[].single_site_location | If you set this parameter, do not set region_location or cross_region_location at the same time. | Yes (one of) | ams03, che01, hkg02, mel01, mex01, mil01, mon01, osl01, par01, sjc04, sao01, seo01, sng01, and tor01 |
serviceid🔗Defines a iam_service_id that can be granted several role based accesss right via attaching iam_policies to it.
serviceid:
+- name: sample-serviceid
+ description: to access ibmcloud services from external
+ servicekeys:
+ - name: primarykey
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the serviceid | Yes | |
| description | short description of the serviceid | No | |
| servicekeys | collection of servicekeys that should be created for the parent serviceid | No | |
| servicekeys.name | Name of the servicekey | Yes |
For Microsoft Azure, the following object type is supported:
Defines an infrastructure configuration onto which OpenShift will be provisioned.
azure:
+- name: sample
+ resource_group:
+ name: sample
+ location: westeurope
+ vnet:
+ name: vnet
+ address_space: 10.0.0.0/22
+ control_plane:
+ subnet:
+ name: control-plane-subnet
+ address_prefixes: 10.0.0.0/23
+ compute:
+ subnet:
+ name: compute-subnet
+ address_prefixes: 10.0.2.0/23
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the azure definition object, will be referenced by openshift | Yes | |
| resource_group | Resource group attributes | Yes | |
| resource_group.name | Name of the resource group (will be provisioned) | Yes | unique value, it must not exist |
| resource_group.location | Azure location | Yes | to pick a different location, run: az account list-locations -o table |
| vnet | Virtual network attributes | Yes | |
| vnet.name | Name of the virtual network | Yes | |
| vnet.address_space | Address space of the virtual network | Yes | |
| control_plane | Control plane (master) nodes attributes | Yes | |
| control_plane.subnet | Control plane nodes subnet attributes | Yes | |
| control_plane.subnet.name | Name of the control plane nodes subnet | Yes | |
| control_plane.subnet.address_prefixes | Address prefixes of the control plane nodes subnet (divided by a , comma, if relevant) | Yes | |
| control_plane.vm | Control plane nodes virtual machine attributes | Yes | |
| control_plane.vm.size | Virtual machine size (aka flavour) of the control plane nodes | Yes | Standard_D8s_v3, Standard_D16s_v3, Standard_D32s_v3 |
| compute | Compute (worker) nodes attributes | Yes | |
| compute.subnet | Compute nodes subnet attributes | Yes | |
| compute.subnet.name | Name of the compute nodes subnet | Yes | |
| compute.subnet.address_prefixes | Address prefixes of the compute nodes subnet (divided by a , comma, if relevant) | Yes | |
| compute.vm | Compute nodes virtual machine attributes | Yes | |
| compute.vm.size | Virtual machine size (aka flavour) of the compute nodes | Yes | See the full list of supported virtual machine sizes |
| compute.vm.disk_size_gb | Disk size in GBs of the compute nodes virtual machine | Yes | minimum value is 128 |
| compute.vm.count | Number of compute nodes virtual machines | Yes | minimum value is 3 |
For Amazon AWS, the following object types are supported:
nfs_server🔗Defines a new Elastic File Storage (EFS) service that is connected to the OpenShift cluster within the same VPC. The file storage will be used as the back-end for the efs-nfs-client OpenShift storage class.
nfs_server:
+- name: sample-elastic
+ infrastructure:
+ aws_region: eu-west-1
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the EFS File System service to be created | Yes | |
| infrastructure | Infrastructure attributes | Yes | |
| infrastructure.aws_region | AWS region where the storage will be provisioned | Yes |
For vSphere, the following object types are supported:
vsphere🔗Defines the vSphere vCenter onto which OpenShift will be provisioned.
vsphere:
+- name: sample
+ vcenter: 10.99.92.13
+ datacenter: Datacenter1
+ datastore: Datastore1
+ cluster: Cluster1
+ network: "VM Network"
+ folder: /Datacenter1/vm/sample
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the vSphere definition, will be referenced by openshift | Yes | |
| vcenter | Host or IP address of the vSphere Center | Yes | |
| datacenter | vSphere Data Center to be used for the virtual machines | Yes | |
| datastore | vSphere Datastore to be used for the virtual machines | Yes | |
| cluster | vSphere cluster to be used for the virtual machines | Yes | |
| resource_pool | vSphere resource pool | No | |
| network | vSphere network to be used for the virtual machines | Yes | |
| folder | Fully qualified folder name into which the OpenShift cluster will be placed; the folder must exist | Yes |
vm_definition🔗Defines the virtual machine properties to be used for the control-plane nodes and compute nodes.
vm_definition:
+- name: control-plane
+ vcpu: 8
+ memory_mb: 32768
+ boot_disk_size_gb: 100
+- name: compute
+ vcpu: 16
+ memory_mb: 65536
+ boot_disk_size_gb: 200
+ # Optional overrides for vsphere properties
+ # datastore: Datastore1
+ # network: "VM Network"
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the VM definition, will be referenced by openshift | Yes | |
| vcpu | Number of virtual CPUs to be assigned to the VMs | Yes | |
| memory_mb | Amount of memory in MiB of the virtual machines | Yes | |
| boot_disk_size_gb | Size of the virtual machine boot disk in GiB | Yes | |
| datastore | vSphere Datastore to be used for the virtual machines, overrides vsphere.datastore | No | |
| network | vSphere network to be used for the virtual machines, overrides vsphere.network | No |
nfs_server🔗Defines an existing NFS server that will be used for the OpenShift NFS storage class.
nfs_server:
+- name: sample-nfs
+ infrastructure:
+ host_ip: 10.99.92.31
+ storage_folder: /data/nfs
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the NFS server | Yes | |
| infrastructure | Infrastructure attributes | Yes | |
| infrastructure.host_ip | Host or IP address of the NFS server | Yes | |
| infrastructure.storage_folder | Folder that holds the data, this will be mounted from the NFS storage class | Yes |
For logging and auditing of Cloud Pak for Data we make use of the OpenShift logging framework, which delivers a lot of flexibility in capturing logs from applications, storing them in an ElasticSearch datastore in the cluster (currently not supported by the deployer), or forwarding the log entries to external log collectors such as an ElasticSearch, Fluentd, Loki and others.
OpenShift logging captures 3 types of logging entries from workload that is running on the cluster:
openshift_logging🔗Defines how OpenShift forwards the logs to external log collectors. Currently, the following log collector types are supported:
When OpenShift logging is activated via the openshift_logging object, all 3 logging types are activated automatically. You can specify logging_output items to forward log records to the log collector of your choice. In the below example, the application logs are forwarded to a loki server https://loki-application.sample.com and audit logs to https://loki-audit.sample.com, both have the same certificate to connect with:
openshift_logging:
+- openshift_cluster_name: pluto-01
+ configure_es_log_store: False
+ cluster_wide_logging:
+ - input: application
+ logging_name: loki-application
+ - input: infrastructure
+ logging_name: loki-application
+ - input: audit
+ logging_name: loki-audit
+ logging_output:
+ - name: loki-application
+ type: loki
+ url: https://loki-application.sample.com
+ certificates:
+ cert: pluto-01-loki-cert
+ key: pluto-01-loki-key
+ ca: pluto-01-loki-ca
+ - name: loki-audit
+ type: loki
+ url: https://loki-audit.sample.com
+ certificates:
+ cert: pluto-01-loki-cert
+ key: pluto-01-loki-key
+ ca: pluto-01-loki-ca
+Cloud Pak for Data and Foundational Services application logs are automatically picked up and forwarded to the loki-application logging destination and no additional configuration is needed.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| openshift_cluster_name | Name of the OpenShift cluster to configure the logging for | Yes | |
| configure_es_log_store | Must internal ElasticSearch log store and Kibana be provisioned? (default False) | No | True, False (default) |
| cluster_wide_logging | Defines which classes of log records will be sent to the log collectors | No | |
| cluster_wide_logging.input | Specifies OpenShift log records class to forwawrd | Yes | application, infrastructure, audit |
| cluster_wide_logging.logging_name | Specifies the logging_output to send the records to . If not specified, records will be sent to the internal log only | No | |
| cluster_wide_logging.labels | Specify your own labels to be added to the log records. Every logging input/output combination can have its own labes | No | |
| logging_output | Defines the log collectors. If configure_es_log_store is True, output will always be sent to the internal ES log store | No | |
| logging_output.name | Log collector name, referenced by cluster_wide_logging or cp4d_audit | Yes | |
| logging_output.type | Type of the log collector, currently only loki is possible | Yes | loki |
| logging_output.url | URL of the log collector; this URL must be reachable from within the cluster | Yes | |
| logging_output.certificates | Defines the vault secrets that hold the certificate elements | Yes, if url is https | |
| logging_output.certificates.cert | Public certificate to connect to the URL | Yes | |
| logging_output.certificates.key | Private key to connect to the URL | Yes | |
| logging_output.certificates.ca | Certificate Authority bundle to connect to the URL | Yes |
If you also want to activate audit logging for Cloud Pak for Data, you can do this by adding a cp4d_audit_config object to your configuration. With the below example, the Cloud Pak for Data audit logger is configured to write log records to the standard output (stdout) of the pods, after which they are forwarded to the loki-audit logging destination by a ClusterLogForwarder custom resource. Optionally labels can be specified which are added to the ClusterLogForwarder custom resource pipeline entry.
cp4d_audit_config:
+- project: cpd
+ audit_replicas: 2
+ audit_output:
+ - type: openshift-logging
+ logging_name: loki-audit
+ labels:
+ cluster_name: "{{ env_id }}"
+Info
Because audit log entries are written to the standard output, they will also be picked up by the generic application log forwarder and will therefore also appear in the application logging destination.
IBM Cloud Pak for Data has a centralized auditing component for base platform and services auditable events. Audit events include login and logout to the platform, creation and deletion of connections and many more. Services that support auditing are documented here: https://www.ibm.com/docs/en/cloud-paks/cp-data/4.0?topic=data-services-that-support-audit-logging
The Cloud Pak Deployer simplifies the recording of audit log entries by means of the OpenShift logging framework, which can in turn be configured to forward entries to various log collectors such as Fluentd, Loki and ElasticSearch.
cp4d_audit_config🔗A cp4d_audit_config entry defines the audit configuration for a Cloud Pak for Data instance (OpenShift project). The main configuration items are the number of replicas and the output. Currently only one output type is supported: openshift-logging, which allows the OpenShift logging framework to pick up audit entries and forward to the designated collectors.
When a cp4d_audit_config entry exists for a certain cp4d project, the zen-audit-config ConfigMap is updated and then the audit logging deployment is restarted. If no configuration changes have been made, no restart is done.
Additionally, for the audit_output entries, the OpenShift logging ClusterLogForwarder instance is updated to forward audit entries to the designated logging output. In the example below the auditing is configured with 2 replicas and an input and pipeline is added to the ClusterLogForwarder instance so output to the matching channel defined in openshift_logging.logging_output.
cp4d_audit_config:
+- project: cpd
+ audit_replicas: 2
+ audit_output:
+ - type: openshift-logging
+ logging_name: loki-audit
+ labels:
+ cluster_name: "{{ env_id }}"
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| project | Name of OpenShift project of the matching cp4d entry. The cp4d project must exist. | Yes | |
| audit_replicas | Number of replicas for the Cloud Pak for Data audit logger. | No (default 1) | |
| audit_output | Defines where the audit logs should be written to | Yes | |
| audit_output.type | Type of auditing output, defines where audit logging entries will be written | Yes | openshift-logging |
| audit_output.logging_name | Name of the logging_output entry in the openshift_logging object. This logging_output entry must exist. | Yes | |
| audit_output.labels | Optional list of labels set to the ClusterLogForwarder custom resource pipeline | No |
For monitoring of Cloud Pak for Data we make use of the OpenShift Monitoring framework. The observations generated by Cloud Pak for Data are pushed to the OpenShift Monitoring Prometheus endpoint. This will allow (external) monitoring tools to combine the observations from the OpenShift platform and Cloud Pak for Data from a single source.
To deploy Cloud Pak for Data Monitors, its is mandatory to also enable the OpenShift monitoring. OpenShift monitoring is activated via the openshift_monitoring object.
openshift_monitoring:
+- openshift_cluster_name: pluto-01
+ user_workload: enabled
+ remote_rewrite_url: http://www.example.com:1234/receive
+ retention_period: 15d
+ pvc_storage_class: ibmc-vpc-block-retain-general-purpose
+ pvc_storage_size_gb: 100
+ grafana_operator: enabled
+ grafana_project: grafana
+ labels:
+ cluster_name: pluto-01
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| user_worload | Allow pushing Prometheus metrics to OpenShift (must be set to True for monitoring to work) | Yes | True, False |
| pvc_storage_class | Storage class to keep persistent monitoring data | No | Valid storage class |
| pvc_storage_size_gb | Size of the PVC holding the monitoring data | Yes if pv_storage_class is set | |
| remote_rewrite_url | Set this value to redirect metrics to remote Prometheus | NO | |
| retention_period | Number of seconds (s), minutes (m), hours(h), days (d), weeks (w), years (y) to retain monitoring data. Default is 15d | Yes | |
| labels | Additional labels to be added to the metrics | No | |
| grafana_operator | Enable Grafana community operator? | No | False (default), True |
| grafana_project | If enabled, project in which to enable the Grafana operator | Yes, if grafana_operator enabled |
Note Labels must be specified as a YAML record where each line is a key-value. The labels will be added to the prometheus key of the user-workload-monitoring-config ConfigMap and to the prometheusK8S key of the cluster-monitoring-config ConfigMap.
Note When the Grafana operator is enabled, you can build your own Grafana dashboard based on the metrics collected by Prometheus. When installed, Grafana creates a local admin user with user name root and passwowrd secret. Grafana can be accessed using the OpenShift route that is created in the project specified by grafana_project.
The observations of Cloud Pak for Data are generated using the zen-watchdog component, which is part of the cpd_platform cartridge and therefore available on each instance of Cloud Pak for Data. Part of the zen-watchdog installation is a set of monitors which focus on the technical deployment of Cloud Pak for Data (e.g. running pods and bound Persistent Volume Claims (pvcs)).
Additional monitors which focus more on the operational usage of Cloud Pak for Data can be deployed as well. These monitors are maintained in a seperate Git repository and be accessed at IBM/cp4d-monitors. Using the Cloud Pak Deployer, monitors can be deployed which uses the Cloud Pak for Data zen-watchdog monitor framework. This allows adding custom monitors to the zen-watchdog, making these custom monitors visible in the Cloud Pak for Data metrics.
Using the Cloud Pak Deployer cp4d_monitors capability implements the following: - Create Cloud Pak for Data ServiceMonitor endpoint to forward zen-watchdog monitor events to OpenShift Cluster monitoring - Create source repository auth secrets (optional, if pulling monitors from secure repo) - Create target container registry auth secrets (optional, if pushing monitor images to secure container registry) - Deploy custom monitors, which will be added to the zen-watchdog monitor framework
For custom monitors to be deployed, it is mandatory to enable the OpenShift user-workload monitoring, as specified in OpenShift monitoring.
The Cloud Pak for Data monitors are specified in a cp4d_monitors definition.
cp4d_monitors:
+- name: cp4d-monitor-set-1
+ cp4d_instance: zen-45
+ openshift_cluster_name: pluto-01
+ default_monitor_source_repo: https://github.com/IBM/cp4d-monitors
+ #default_monitor_source_token_secret: monitors_source_repo_secret
+ #default_monitor_target_cr: de.icr.io/monitorrepo
+ #default_monitor_target_cr_user_secret: monitors_target_cr_username
+ #default_monitor_target_cr_password_secret: monitors_target_cr_password
+ # List of monitors
+ monitors:
+ - name: cp4dplatformcognosconnectionsinfo
+ context: cp4d-cognos-connections-info
+ label: latest
+ schedule: "*/15 * * * *"
+ - name: cp4dplatformcognostaskinfo
+ context: cp4d-cognos-task-info
+ label: latest
+ schedule: "*/15 * * * *"
+ - name: cp4dplatformglobalconnections
+ context: cp4d-platform-global-connections
+ label: latest
+ schedule: "*/15 * * * *"
+ - name: cp4dplatformwatsonstudiojobinfo
+ context: cp4d-watsonstudio-job-info
+ label: latest
+ schedule: "*/15 * * * *"
+ - name: cp4dplatformwatsonstudiojobscheduleinfo
+ context: cp4d-watsonstudio-job-schedule-info
+ label: latest
+ schedule: "*/15 * * * *"
+ - name: cp4dplatformwatsonstudioruntimeusage
+ context: cp4d-watsonstudio-runtime-usage
+ label: latest
+ schedule: "*/15 * * * *"
+ - name: cp4dplatformwatsonknowledgecataloginfo
+ context: cp4d-wkc-info
+ label: latest
+ schedule: "*/15 * * * *"
+ - name: cp4dplatformwmldeploymentspaceinfo
+ context: cp4d-wml-deployment-space-info
+ label: latest
+ schedule: "*/15 * * * *"
+ - name: cp4dplatformwmldeploymentspacejobinfo
+ context: cp4d-wml-deployment-space-job-info
+ label: latest
+ schedule: "*/15 * * * *"
+Each cp4d_monitors entry contains a set of default settings, which are applicable to the monitors list. These defaults can be overwritten per monitor if needed.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | The name of the monitor set | Yes | lowercase RFC 1123 subdomain (1) |
| cp4d_instance | The OpenShift project (namespace) on which the Cloud Pak for Data instance resides | Yes | |
| openshift_cluster_name | The Openshift cluster name | Yes | |
| default_monitor_source_repo | The default repository location of all monitors located in the monitors section | No | |
| default_monitor_source_token_secret | The default repo access token secret name, must be available in the vault | No | |
| default_monitor_target_cr | The default target container registry (cr) for the monitor image to be pushed. When omitted, the OpenShift internal registry is used | No | |
| default_monitor_target_cr_user_secret | The default target container registry user name secret name used to push the monitor image. Must be available in the vault | No | |
| default_monitor_target_cr_password_secret | The default target container registry password secret name used to push the monitor image. Must be available in the vault | No | |
| monitors | List of monitors | Yes |
Per monitors entry, the following settings are specified:
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | The name of the monitor entry | Yes | lowercase RFC 1123 subdomain (1) |
| monitor_source_repo | Overrides default_monitor_source_repo for this single monitor | No | |
| monitor_source_token_secret | Overrides default_monitor_source_token_secret for this single monitor | No | |
| monitor_target_cr | Overrides default_monitor_target_cr for this single monitor | No | |
| monitor_target_cr_user_secret | Overrides default_monitor_target_cr_user_secret for this single monitor | No | |
| monitor_target_cr_user_password | Overrides default_monitor_target_cr_user_password for this single monitor | No | |
| context | Sets the context of the monitor the the source repo (sub folder name) | Yes | |
| label | Set the label of the pushed image, default to 'latest' | No | |
| schedule | Sets the schedule of the generated Cloud Pak for Data monitor cronjob | Yes |
Each monitor has a set of event_types, which contain the observations generated by the monitor. These event types are retrieved directly from the github repository, which it is expected that each context contains a file called event_types.yml. During deployment of the monitor this file is retrieved and used to populate the event_types of the monitor.
If the Deployer runs and the monitor is already deployed, the following process is used: - The build process is restarted to ensure the latest image of monitor is used - A comparison is made between the monitor's current configuration and the configuration created by the Deployer. If these are identical, the monitor's configuration is left as-is, however if these are different, the monitor's configuration is rebuild and the monitor is re-deployed.
This monitor counts the number of Global Platform connections and for each Global Platform Connection a test is executed to test whether the connection can still be established.
Once the monitor is deployed, the following metrics are available in IBM Cloud Pak for Data.
On the Platform Management Events page the following entries are added: - Cloud Pak for Data Global Connections Count - Global Connection - <Global Connection Name> (for each connection)
https://<CP4D-BASE-URL>/zen/metrics
It will generate 2 types of metrics:
# HELP global_connections_count
+# TYPE global_connections_count gauge
+global_connections_count{event_type="global_connections_count",monitor_type="cp4d_platform_global_connections",reference="Cloud Pak for Data Global Connections Count"} 2
+
+# HELP global_connection_valid
+# TYPE global_connection_valid gauge
+global_connection_valid{event_type="global_connection_valid",monitor_type="cp4d_platform_global_connections",reference="Cognos MetaStore Connection"} 1
+global_connection_valid{event_type="global_connection_valid",monitor_type="cp4d_platform_global_connections",reference="Cognos non-shared"} 0
+Zen Watchdog metrics (used in platform management events) - watchdog_cp4d_platform_global_connections_global_connections_count - watchdog_cp4d_platform_global_connections_global_connection_valid (for each connection)
Zen Watchdog metrics can have the following values: - 2 (info) - 1 (warning) - 0 (critical)
# HELP watchdog_cp4d_platform_global_connections_global_connection_valid
+# TYPE watchdog_cp4d_platform_global_connections_global_connection_valid gauge
+watchdog_cp4d_platform_global_connections_global_connection_valid{event_type="global_connection_valid",monitor_type="cp4d_platform_global_connections",reference="Cognos MetaStore Connection"} 2
+watchdog_cp4d_platform_global_connections_global_connection_valid{event_type="global_connection_valid",monitor_type="cp4d_platform_global_connections",reference="Cognos non-shared"} 1
+
+# HELP watchdog_cp4d_platform_global_connections_global_connections_count
+# TYPE watchdog_cp4d_platform_global_connections_global_connections_count gauge
+watchdog_cp4d_platform_global_connections_global_connections_count{event_type="global_connections_count",monitor_type="cp4d_platform_global_connections",reference="Cloud Pak for Data Global Connections Count"} 2
+You can configure one or more OpenShift clusters that will be layed down on the specified infrastructure, or which already exist.
Dependent on the cloud platform on which the OpenShift cluster will be provisioned, different installation methods apply. For IBM Cloud, Terraform is used, whereas for vSphere the IPI installer is used. On AWS (ROSA), the rosa CLI is used to create and modify ROSA clusters. Each of the different platforms have slightly different properties for the openshift objects.
openshift🔗For OpenShift, there are 5 flavours:
Every OpenShift cluster definition of a few mandatory properties that control which version of OpenShift is installed, the number and flavour of control plane and compute nodes and the underlying infrastructure, dependent on the cloud platform on which it is provisioned. Storage is a mandatory element for every openshift definition. For a list of supported storage types per cloud platform, refer to Supported storage types.
Additionally, one can configure Upstream DNS Servers and OpenShift logging.
The Multicloud Object Gateway (MCG) supports access to s3-compatible object storage via an underpinning block/file storage class, through the Noobaa operator. Some Cloud Pak for Data services such as Watson Assistant need object storage to run. MCG does not need to be installed if OpenShift Data Foundation (fka OCS) is also installed as the operator includes Noobaa.
When using the Cloud Pak Deployer on an existing OpenShift cluster, the scripts assume that the cluster is already operational and that any storage classes have been pre-created. The deployer accesses the cluster through a vault secret with the kubeconfig information; the name of the secret is <name>-kubeconfig.
openshift:
+- name: sample
+ ocp_version: 4.8
+ cluster_name: sample
+ domain_name: example.com
+ cloud_native_toolkit: False
+ oadp: False
+ infrastructure:
+ type: standard
+ processor_architecture: amd64
+ upstream_dns:
+ - name: sample-dns
+ zones:
+ - example.com
+ dns_servers:
+ - 172.31.2.73:53
+ gpu:
+ install: auto
+ openshift_ai:
+ install: auto
+ channel: auto
+ mcg:
+ install: True
+ storage_type: storage-class
+ storage_class: managed-nfs-storage
+ openshift_storage:
+ - storage_name: nfs-storage
+ storage_type: nfs
+ # ocp_storage_class_file: managed-nfs-storage
+ # ocp_storage_class_block: managed-nfs-storage
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the OpenShift cluster | Yes | |
| ocp_version | OpenShift version of the cluster, used to download the client. If you want to install 4.10, specify "4.10" | Yes | >= 4.6 |
| cluster_name | Name of the cluster (part of the FQDN) | Yes | |
| domain_name | Domain name of the cluster (part of the FQDN) | Yes | |
| cloud_native_toolkit | Must the Cloud Native Toolkit (OpenShift GitOps) be installed? | No | True, False (default) |
| oadp | Must the OpenShift Advanced Data Protection operator be installed | No | True, False (default) |
| infrastructure.type | Infrastructure OpenShift is deployed on. See below for additional explanation | detect (default) | |
| infrastructure.processor_architecture | Architecture of the processor that the OpenShift cluster is deployed on | No | amd64 (default), ppc64le, s390x |
| openshift_logging[] | Logging attributes for OpenShift cluster, see OpenShift logging | No | |
| upstream_dns[] | Upstream DNS servers(s), see Upstream DNS Servers | No | |
| gpu | Control Node Feature Discovery and NVIDIA GPU operators | No | |
| gpu.install | Must Node Feature Discovery and NVIDIA GPU operators be installed (Once installed, False does not uninstall). auto will install the operators if needed by any of the Cloud Pak/watsonx components | Yes | auto, True, False |
| openshift_ai | Control installation of OpenShift AI | No | |
| openshift_ai.install | Must OpenShift AI be installed (Once installed, False does not uninstall). auto will install OpenShift AI if needed by any of the Cloud Pak/watsonx components | Yes | auto, True, False |
| openshift_ai.channel | Which oeprator channel must be installed | No | auto (default), stable, … |
| mcg | Multicloud Object Gateway properties | No | |
| mcg.install | Must Multicloud Object Gateway be installed (Once installed, False does not uninstall) | Yes | True, False |
| mcg.storage_type | Type of storage supporting the object Noobaa object storage | Yes | storage-class |
| mcg.storage_class | Storage class supporting the Noobaa object storage | Yes | Existing storage class |
| openshift_storage[] | List of storage definitions to be defined on OpenShift, see below for further explanation | Yes |
When deploying on existing OpenShift, the underlying infrastructure can pose some restrictions on capabilities available. For example, Red Hat OpenShift on IBM Cloud (aka ROKS) does not include the Machine Config Operator and ROSA on AWS does not allow to set labels for Machine Config Pools. This means that node settings required for Cloud Pak for Data must be applied in a non-standard manner.
The following values are allowed for infrastructure.type:
detect (default): The deployer will attempt to detect the underlying cloud infrastructure. This is done by retrieving the existing storage classes and then inferring the cloud type.standard: The deployer will assume a standard OpenShift cluster with no further restrictions. This is the fallback value for detect if the underlying infra cannot be detected. aws-self-managed: A self-managed OpenShift cluster on AWS. No restrictions.aws-rosa: Managed Red Hat OpenShift on AWS. Some restrictions with regards to Machine Config Pools apply.azure-aro: Managed Red Hat OpenShift on Azure. No known restrictions.vsphere: OpenShift on vSphere. No known restrictions.| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| storage_name | Name of the storage definition, to be referenced by the Cloud Pak | Yes | |
| storage_type | Type of storage class to use in the OpenShift cluster | Yes | nfs, ocs, aws-elastic, auto, custom |
| ocp_storage_class_file | OpenShift storage class to use for file storage if different from default for storage_type | Yes if storage_type is custom | |
| ocp_storage_class_block | OpenShift storage class to use for block storage if different from default for storage_type | Yes if storage_type is custom |
Info
The custom storage_type can be used in case you want to use a non-standard storage class(es). In this case the storage class(es) must be already configured on the OCP cluster and set in the respective ocp_storage_class_file and ocp_storage_class_block variables
Info
The auto storage_type will let the deployer automatically detect the storage type based on the existing storage classes in the OpenShift cluster.
An openshift definition always includes the type(s) of storage that it will provide. When the OpenShift cluster is provisioned by the deployer, the necessary infrastructure and storage class(es) are also configured. In case an existing OpenShift cluster is referenced by the configuration, the storage classes are expected to exist already.
The table below indicates which storage classes are supported by the Cloud Pak Deployer per cloud infrastructure.
Warning
The ability to provision or use certain storage types does not imply support by the Cloud Paks or by OpenShift itself. There are several restrictions for production use OpenShift Data Foundation, for example when on ROSA.
| Cloud Provider | NFS Storage | OCS/ODF Storage | Portworx | Elastic | Custom (2) |
|---|---|---|---|---|---|
| ibm-cloud | Yes | Yes | Yes | No | Yes |
| vsphere | Yes (1) | Yes | No | No | Yes |
| aws | No | Yes | No | Yes (3) | Yes |
| azure | No | Yes | No | No | Yes |
| existing-ocp | Yes | Yes | No | Yes | Yes |
managed-nfs-storage storage class. The deployer will not provision or change the NFS server itself.custom storage type, you must specify the storage class to be used for block (RWO) and file (RWX) storage.nfs_server object is required to define the "file server" storage on AWS.VPC-based OpenShift cluster on IBM Cloud, using the Red Hat OpenShift Kubernetes Services (ROKS).
openshift:
+- name: sample
+ managed: True
+ ocp_version: 4.8
+ compute_flavour: bx2.16x64
+ secondary_storage: 900gb.10iops-tier
+ compute_nodes: 3
+ cloud_native_toolkit: False
+ oadp: False
+ infrastructure:
+ type: vpc
+ vpc_name: sample
+ subnets:
+ - sample-subnet-zone-1
+ - sample-subnet-zone-2
+ - sample-subnet-zone-3
+ cos_name: sample-cos
+ private_only: False
+ deny_node_ports: False
+ upstream_dns:
+ - name: sample-dns
+ zones:
+ - example.com
+ dns_servers:
+ - 172.31.2.73:53
+ mcg:
+ install: True
+ storage_type: storage-class
+ storage_class: managed-nfs-storage
+ openshift_ai:
+ install: auto
+ channel: auto
+ openshift_storage:
+ - storage_name: nfs-storage
+ storage_type: nfs
+ nfs_server_name: sample-nfs
+ - storage_name: ocs-storage
+ storage_type: ocs
+ storage_flavour: bx2.16x64
+ secondary_storage: 900gb.10iops-tier
+ ocs_storage_label: ocs
+ ocs_storage_size_gb: 500
+ ocs_version: 4.8.0
+ - storage_name: pwx-storage
+ storage_type: pwx
+ pwx_etcd_location: {{ ibm_cloud_region }}
+ pwx_storage_size_gb: 200
+ pwx_storage_iops: 10
+ pwx_storage_profile: "10iops-tier"
+ stork_version: 2.6.2
+ portworx_version: 2.7.2
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the OpenShift cluster | Yes | |
| managed | Is the ROKS cluster managed by this deployer? See note below. | No | True (default), False |
| ocp_version | ROKS Kubernetes version. If you want to install 4.10, specify "4.10" | Yes | >= 4.6 |
| compute_flavour | Type of compute node to be used | Yes | Node flavours |
| secondary_storage | Additional storage to be added to the compute servers | No | 900gb.10iops-tier, … |
| compute_nodes | Total number of compute nodes. This must be a factor of the number of subnets | Yes | Integer |
| resource_group | IBM Cloud resource group for the ROKS cluster | Yes | |
| cloud_native_toolkit | Must the Cloud Native Toolkit (OpenShift GitOps) be installed? | No | True, False (default) |
| oadp | Must the OpenShift Advanced Data Protection operator be installed | No | True, False (default) |
| infrastructure.type | Type of infrastructure to provision ROKS cluster on | No | vpc |
| infrastructure.vpc_name | Name of the VPC if type is vpc | Yes, inferrred from vpc | Existing VPC |
| infrastructure.subnets | List of subnets within the VPC to use. Either 1 or 3 subnets must be specified | Yes | Existing subnet |
| infrastructure.cos_name | Reference to the cos object created for this cluster | Yes | Existing cos object |
| infrastructure.private_only | If true, it indicates that the ROKS cluster must be provisioned without public endpoints | No | True, False (default) |
| infrastructure.deny_node_ports | If true, the Allow ICMP, TCP and UDP rules for the security group associated with the ROKS cluster are removed if present. If false, the Allow ICMP, TCP and UDP rules are added if not present. | No | True, False (default) |
| openshift_logging[] | Logging attributes for OpenShift cluster, see OpenShift logging | No | |
| upstream_dns[] | Upstream DNS servers(s), see Upstream DNS Servers | No | |
| gpu | Control Node Feature Discovery and NVIDIA GPU operators | No | |
| gpu.install | Must Node Feature Discovery and NVIDIA GPU operators be installed (Once installed, False does not uninstall). auto will install the operators if needed by any of the Cloud Pak/watsonx components | Yes | auto, True, False |
| openshift_ai | Control installation of OpenShift AI | No | |
| openshift_ai.install | Must OpenShift AI be installed (Once installed, False does not uninstall). auto will install OpenShift AI if needed by any of the Cloud Pak/watsonx components | Yes | auto, True, False |
| openshift_ai.channel | Which oeprator channel must be installed | No | auto (default), stable, … |
| mcg | Multicloud Object Gateway properties | No | |
| mcg.install | Must Multicloud Object Gateway be installed (Once installed, False does not uninstall) | Yes | True, False |
| mcg.storage_type | Type of storage supporting the object Noobaa object storage | Yes | storage-class |
| mcg.storage_class | Storage class supporting the Noobaa object storage | Yes | Existing storage class |
| openshift_storage[] | List of storage definitions to be defined on OpenShift, see below for further explanation | Yes |
The managed attribute indicates whether the ROKS cluster is managed by the Cloud Pak Deployer. If set to False, the deployer will not provision the ROKS cluster but expects it to already be available in the VPC. You can still use the deployer to create the VPC, the subnets, NFS servers and other infrastructure, but first run it without an openshift element. Once the VPC has been created, manually create an OpenShift cluster in the VPC and then add the openshift element with managed set to False. If you intend to use OpenShift Container Storage, you must also activate the add-on and create the OcsCluster custom resource.
Warning
If you set infrastructure.private_only to True, the server from which you run the deployer must be able to access the ROKS cluster via its private endpoint, either by establishing a VPN to the cluster's VPC, or by making sure the deployer runs on a server that has a connection with the ROKS VPC via a transit gateway.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| openshift_storage[] | List of storage definitions to be defined on OpenShift | Yes | |
| storage_name | Name of the storage definition, to be referenced by the Cloud Pak | Yes | |
| storage_type | Type of storage class to create in the OpenShift cluster | Yes | nfs, ocs or pwx |
| storage_flavour | Type of compute node to be used for the storage nodes | Yes | Node flavours, default is bx2.16x64 |
| secondary_storage | Additional storage to be added to the storage server | No | 900gb.10iops-tier, … |
| nfs_server_name | Name of the NFS server within the VPC | Yes if storage_type is nfs | Existing nfs_server |
| ocs_storage_label | Label to be used for the dedicated OCS nodes in the cluster | Yes if storage_type is ocs | |
| ocs_storage_size_gb | Size of the OCS storage in Gibibytes (Gi) | Yes if storage_type is ocs | |
| ocs_version | Version of OCS (ODF) to be deployed. If left empty, the latest version will be deployed | No | >= 4.6 |
| pwx_etcd_location | Location where the etcd service will be deployed, typically the same region as the ROKS cluster | Yes if storage_type is pwx | |
| pwx_storage_size_gb | Size of the Portworx storage that will be provisioned | Yes if storage_type is pwx | |
| pwx_storage_iops | IOPS for the storage volumes that will be provisioned | Yes if storage_type is pwx | |
| pwx_storage_profile | IOPS storage tier the storage volumes that will be provisioned | Yes if storage_type is pwx | |
| stork_version | Version of the Portworx storage orchestration layer for Kubernetes | Yes if storage_type is pwx | |
| portworx_version | Version of the Portworx storage provider | Yes if storage_type is pwx |
Warning
When deploying a ROKS cluster with OpenShift Data Foundation (fka OpenShift Container Storage/OCS), the minimum version of OpenShift is 4.7.
openshift:
+- name: sample
+ domain_name: example.com
+ vsphere_name: sample
+ ocp_version: 4.8
+ control_plane_nodes: 3
+ control_plane_vm_definition: control-plane
+ compute_nodes: 3
+ compute_vm_definition: compute
+ api_vip: 10.99.92.51
+ ingress_vip: 10.99.92.52
+ cloud_native_toolkit: False
+ oadp: False
+ infrastructure:
+ openshift_cluster_network_cidr: 10.128.0.0/14
+ upstream_dns:
+ - name: sample-dns
+ zones:
+ - example.com
+ dns_servers:
+ - 172.31.2.73:53
+ gpu:
+ install: auto
+ openshift_ai:
+ install: auto
+ channel: auto
+ mcg:
+ install: True
+ storage_type: storage-class
+ storage_class: thin
+ openshift_storage:
+ - storage_name: nfs-storage
+ storage_type: nfs
+ nfs_server_name: sample-nfs
+ - storage_name: ocs-storage
+ storage_type: ocs
+ ocs_storage_label: ocs
+ ocs_storage_size_gb: 512
+ ocs_dynamic_storage_class: thin
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the OpenShift cluster | Yes | |
| domain_name | Domain name of the cluster, this will also depict the route to the API and ingress endpoints | Yes | |
| ocp_version | OpenShift version. If you want to install 4.10, specify "4.10" | Yes | >= 4.6 |
| control_plane_nodes | Total number of control plane nodes, typically 3 | Yes | Integer |
| control_plane_vm_definition | vm_definition object that will be used to define number of vCPUs and memory for the control plane nodes | Yes | Existing vm_definition |
| compute_nodes | Total number of compute nodes | Yes | Integer |
| compute_vm_definition | vm_definition object that will be used to define number of vCPUs and memory for the compute nodes | Yes | Existing vm_definition |
| api_vip | Virtual IP address that the installer will provision for the API server | Yes | |
| ingress_vip | Virtual IP address that the installer will provision for the ingress server | Yes | |
| cloud_native_toolkit | Must the Cloud Native Toolkit (OpenShift GitOps) be installed? | No | True, False (default) |
| oadp | Must the OpenShift Advanced Data Protection operator be installed | No | True, False (default) |
| infrastructure | Infrastructure properties | No | |
| infrastructure.openshift_cluster_network_cidr | Network CIDR used by the OpenShift pods. Normally you would not have to change this, unless other systems in the network are in the 10.128.0.0/14 subnet. | No | CIDR |
| openshift_logging[] | Logging attributes for OpenShift cluster, see OpenShift logging | No | |
| upstream_dns[] | Upstream DNS servers(s), see Upstream DNS Servers | No | |
| gpu | Control Node Feature Discovery and NVIDIA GPU operators | No | |
| gpu.install | Must Node Feature Discovery and NVIDIA GPU operators be installed (Once installed, False does not uninstall). auto will install the operators if needed by any of the Cloud Pak/watsonx | Yes | auto, True, False |
| openshift_ai | Control installation of OpenShift AI | No | |
| openshift_ai.install | Must OpenShift AI be installed (Once installed, False does not uninstall). auto will install OpenShift AI if needed by any of the Cloud Pak/watsonx components | Yes | auto, True, False |
| openshift_ai.channel | Which oeprator channel must be installed | No | auto (default), stable, … |
| mcg | Multicloud Object Gateway properties | No | |
| mcg.install | Must Multicloud Object Gateway be installed (Once installed, False does not uninstall) | Yes | True, False |
| mcg.storage_type | Type of storage supporting the object Noobaa object storage | Yes | storage-class |
| mcg.storage_class | Storage class supporting the Noobaa object storage | Yes | Existing storage class |
| openshift_storage[] | List of storage definitions to be defined on OpenShift, see below for further explanation | Yes |
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| openshift_storage[] | List of storage definitions to be defined on OpenShift | Yes | |
| storage_name | Name of the storage definition, to be referenced by the Cloud Pak | Yes | |
| storage_type | Type of storage class to create in the OpenShift cluster | Yes | nfs or ocs |
| nfs_server_name | Name of the NFS server within the VPC | Yes if storage_type is nfs | Existing nfs_server |
| ocs_version | Version of the OCS operator. If not specified, this will default to the ocp_version | No | >= 4.6 |
| ocs_storage_label | Label to be used for the dedicated OCS nodes in the cluster | Yes if storage_type is ocs | |
| ocs_storage_size_gb | Size of the OCS storage in Gibibytes (Gi) | Yes if storage_type is ocs | |
| ocs_dynamic_storage_class | Storage class that will be used for provisioning OCS. On vSphere clusters, thin is usually available after OpenShift installation | Yes if storage_type is ocs | |
| storage_vm_definition | VM Definition that defines the virtual machine attributes for the OCS nodes | Yes if storage_type is ocs |
nfs_server:
+- name: sample-elastic
+ infrastructure:
+ aws_region: eu-west-1
+
+openshift:
+- name: sample
+ ocp_version: 4.10.34
+ domain_name: cp-deployer.eu
+ compute_flavour: m5.4xlarge
+ compute_nodes: 3
+ cloud_native_toolkit: False
+ oadp: False
+ infrastructure:
+ type: self-managed
+ aws_region: eu-central-1
+ multi_zone: True
+ credentials_mode: Manual
+ private_only: True
+ machine_cidr: 10.2.1.0/24
+ openshift_cluster_network_cidr: 10.128.0.0/14
+ subnet_ids:
+ - subnet-06bbef28f585a0dd3
+ - subnet-0ea5ac344c0fbadf5
+ hosted_zone_id: Z08291873MCIC4TMIK4UP
+ ami_id: ami-09249dd86b1933dd5
+ mcg:
+ install: True
+ storage_type: storage-class
+ storage_class: gp3-csi
+ openshift_storage:
+ - storage_name: ocs-storage
+ storage_type: ocs
+ ocs_storage_label: ocs
+ ocs_storage_size_gb: 512
+ - storage_name: sample-elastic
+ storage_type: aws-elastic
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the OpenShift cluster | Yes | |
| ocp_version | OpenShift version version, specified as x.y.z | Yes | >= 4.6 |
| domain_name | Base domain name of the cluster. Together with the name, this will be the domain of the OpenShift cluster. | Yes | |
| control_plane_flavour | Flavour of the AWS servers used for the control plane nodes. m5.xxlarge is the recommended value 4 GB of memory | Yes | |
| control_plane_nodes | Total number of control plane | Yes | Integer |
| compute_flavour | Flavour of the AWS servers used for the compute nodes. m5.4xlarge is a large node with 16 cores and 64 GB of memory | Yes | |
| compute_nodes | Total number of compute nodes | Yes | Integer |
| cloud_native_toolkit | Must the Cloud Native Toolkit (OpenShift GitOps) be installed? | No | True, False (default) |
| oadp | Must the OpenShift Advanced Data Protection operator be installed | No | True, False (default) |
| infrastructure | Infrastructure properties | Yes | |
| infrastructure.type | Type of OpenShift cluster on AWS. | Yes | rosa or self-managed |
| infrastructure.aws_region | Region of AWS where cluster is deployed. | Yes | |
| infrastructure.multi_zone | Determines whether the OpenShift cluster is deployed across multiple availability zones. Default is True. | No | True (default), False |
| infrastructure.credentials_mode | Security requirement of the Cloud Credential Operator (COO) when doing installations with temporary AWS security credentials. Default (omit) is automatically handled by CCO. | No | Manual, Mint |
| infrastructure.machine_cdr | Machine CIDR. This value will be used to create the VPC and its subnets. In case of an existing VPC, specify the CIDR of that VPC. | No | CIDR |
| infrastructure.openshift_cluster_network_cidr | Network CIDR used by the OpenShift pods. Normally you would not have to change this, unless other systems in the network are in the 10.128.0.0/14 subnet. | No | CIDR |
| infrastructure.subnet_ids | Existing public and private subnet IDs in the VPC to be used for the OpenShift cluster. Must be specified in combination with machine_cidr and hosted_zone_id. | No | Existing subnet IDs |
| infrastructure.private_only | Indicates whether the OpenShift can be accessed from the internet. Default is True | No | True, False |
| infrastructure.hosted_zone_id | ID of the AWS Route 53 hosted zone that controls the DNS entries. If not specified, the OpenShift installer will create a hosted zone for the specified domain_name. This attribute is only needed if you create the OpenShift cluster in an existing VPC | No | |
| infrastructure.control_plane_iam_role | If not standard, specify the IAM role that the OpenShift installer must use for the control plane nodes during cluster creation | No | |
| infrastructure.compute_iam_role | If not standard, specify the IAM role that the OpenShift installer must use for the compute nodes during cluster creation | No | |
| infrastructure.ami_id | ID of the AWS AMI to boot all images | No | |
| openshift_logging[] | Logging attributes for OpenShift cluster, see OpenShift logging | No | |
| gpu | Control Node Feature Discovery and NVIDIA GPU operators | No | |
| gpu.install | Must Node Feature Discovery and NVIDIA GPU operators be installed (Once installed, False does not uninstall). auto will install the operators if needed by any of the Cloud Pak/watsonx | Yes | auto, True, False |
| openshift_ai | Control installation of OpenShift AI | No | |
| openshift_ai.install | Must OpenShift AI be installed (Once installed, False does not uninstall). auto will install OpenShift AI if needed by any of the Cloud Pak/watsonx components | Yes | auto, True, False |
| openshift_ai.channel | Which oeprator channel must be installed | No | auto (default), stable, … |
| mcg | Multicloud Object Gateway properties | No | |
| mcg.install | Must Multicloud Object Gateway be installed (Once installed, False does not uninstall) | Yes | True, False |
| mcg.storage_type | Type of storage supporting the object Noobaa object storage | Yes | storage-class |
| mcg.storage_class | Storage class supporting the Noobaa object storage | Yes | Existing storage class |
| openshift_storage[] | List of storage definitions to be defined on OpenShift, see below for further explanation | Yes |
When deploying the OpenShift cluster within an existing VPC, you must specify the machine_cidr that covers all subnets and the subnet IDs within the VPC. For example:
machine_cidr: 10.243.0.0/24
+ subnets_ids:
+ - subnet-0e63f662bb1842e8a
+ - subnet-0673351cd49877269
+ - subnet-00b007a7c2677cdbc
+ - subnet-02b676f92c83f4422
+ - subnet-0f1b03a02973508ed
+ - subnet-027ca7cc695ce8515
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| openshift_storage[] | List of storage definitions to be defined on OpenShift | Yes | |
| storage_name | Name of the storage definition, to be referenced by the Cloud Pak | Yes | |
| storage_type | Type of storage class to create in the OpenShift cluster | Yes | ocs, aws-elastic |
| ocs_version | Version of the OCS operator. If not specified, this will default to the ocp_version | No | |
| ocs_storage_label | Label to be used for the dedicated OCS nodes in the cluster | Yes if storage_type is ocs | |
| ocs_storage_size_gb | Size of the OCS storage in Gibibytes (Gi) | Yes if storage_type is ocs | |
| ocs_dynamic_storage_class | Storage class that will be used for provisioning ODF. gp3-csi is usually available after OpenShift installation | No |
nfs_server:
+- name: sample-elastic
+ infrastructure:
+ aws_region: eu-west-1
+
+openshift:
+- name: sample
+ ocp_version: 4.10.34
+ compute_flavour: m5.4xlarge
+ compute_nodes: 3
+ cloud_native_toolkit: False
+ oadp: False
+ infrastructure:
+ type: rosa
+ aws_region: eu-central-1
+ multi_zone: True
+ use_sts: False
+ credentials_mode: Manual
+ upstream_dns:
+ - name: sample-dns
+ zones:
+ - example.com
+ dns_servers:
+ - 172.31.2.73:53
+ gpu:
+ install: auto
+ openshift_ai:
+ install: auto
+ channel: auto
+ mcg:
+ install: True
+ storage_type: storage-class
+ storage_class: gp3-csi
+ openshift_storage:
+ - storage_name: ocs-storage
+ storage_type: ocs
+ ocs_storage_label: ocs
+ ocs_storage_size_gb: 512
+ - storage_name: sample-elastic
+ storage_type: aws-elastic
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the OpenShift cluster | Yes | |
| ocp_version | OpenShift version version, specified as x.y.z | Yes | >= 4.6 |
| compute_flavour | Flavour of the AWS servers used for the compute nodes. m5.4xlarge is a large node with 16 cores and 64 GB of memory | Yes | |
| cloud_native_toolkit | Must the Cloud Native Toolkit (OpenShift GitOps) be installed? | No | True, False (default) |
| oadp | Must the OpenShift Advanced Data Protection operator be installed | No | True, False (default) |
| infrastructure | Infrastructure properties | Yes | |
| infrastructure.type | Type of OpenShift cluster on AWS. | Yes | rosa or self-managed |
| infrastructure.aws_region | Region of AWS where cluster is deployed. | Yes | |
| infrastructure.multi_zone | Determines whether the OpenShift cluster is deployed across multiple availability zones. Default is True. | No | True (default), False |
| infrastructure.use_sts | Determines whether AWS Security Token Service must be used by the ROSA installer. Default is False. | No | True, False (default) |
| infrastructure.credentials_mode | Change the security requirement of the Cloud Credential Operator (COO). Default (omit) is automatically handled by CCO. | No | Manual, Mint |
| infrastructure.machine_cdr | Machine CIDR, for example 10.243.0.0/16. | No | CIDR |
| infrastructure.subnet_ids | Existing public and private subnet IDs in the VPC to be used for the OpenShift cluster. Must be specified in combination with machine_cidr. | No | Existing subnet IDs |
| compute_nodes | Total number of compute nodes | Yes | Integer |
| upstream_dns[] | Upstream DNS servers(s), see Upstream DNS Servers | No | |
| openshift_logging[] | Logging attributes for OpenShift cluster, see OpenShift logging | No | |
| upstream_dns[] | Upstream DNS servers(s), see Upstream DNS Servers | No | |
| gpu | Control Node Feature Discovery and NVIDIA GPU operators | No | |
| gpu.install | Must Node Feature Discovery and NVIDIA GPU operators be installed (Once installed, False does not uninstall). auto will install the operators if needed by any of the Cloud Pak/watsonx | Yes | auto, True, False |
| openshift_ai | Control installation of OpenShift AI | No | |
| openshift_ai.install | Must OpenShift AI be installed (Once installed, False does not uninstall). auto will install OpenShift AI if needed by any of the Cloud Pak/watsonx components | Yes | auto, True, False |
| openshift_ai.channel | Which oeprator channel must be installed | No | auto (default), stable, … |
| mcg | Multicloud Object Gateway properties | No | |
| mcg.install | Must Multicloud Object Gateway be installed (Once installed, False does not uninstall) | Yes | True, False |
| mcg.storage_type | Type of storage supporting the object Noobaa object storage | Yes | storage-class |
| mcg.storage_class | Storage class supporting the Noobaa object storage | Yes | Existing storage class |
| openshift_storage[] | List of storage definitions to be defined on OpenShift, see below for further explanation | Yes |
When deploying the OpenShift cluster within an existing VPC, you must specify the machine_cidr that covers all subnets and the subnet IDs within the VPC. For example:
machine_cidr: 10.243.0.0/24
+ subnets_ids:
+ - subnet-0e63f662bb1842e8a
+ - subnet-0673351cd49877269
+ - subnet-00b007a7c2677cdbc
+ - subnet-02b676f92c83f4422
+ - subnet-0f1b03a02973508ed
+ - subnet-027ca7cc695ce8515
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| openshift_storage[] | List of storage definitions to be defined on OpenShift | Yes | |
| storage_name | Name of the storage definition, to be referenced by the Cloud Pak | Yes | |
| storage_type | Type of storage class to create in the OpenShift cluster | Yes | ocs, aws-elastic |
| ocs_version | Version of the OCS operator. If not specified, this will default to the ocp_version | No | |
| ocs_storage_label | Label to be used for the dedicated OCS nodes in the cluster | Yes if storage_type is ocs | |
| ocs_storage_size_gb | Size of the OCS storage in Gibibytes (Gi) | Yes if storage_type is ocs | |
| ocs_dynamic_storage_class | Storage class that will be used for provisioning ODF. gp3-csi is usually available after OpenShift installation | No |
openshift:
+- name: sample
+ azure_name: sample
+ domain_name: example.com
+ ocp_version: 4.10.54
+ cloud_native_toolkit: False
+ oadp: False
+ network:
+ pod_cidr: "10.128.0.0/14"
+ service_cidr: "172.30.0.0/16"
+ gpu:
+ install: auto
+ openshift_ai:
+ install: auto
+ channel: auto
+ openshift_storage:
+ - storage_name: ocs-storage
+ storage_type: ocs
+ ocs_storage_label: ocs
+ ocs_storage_size_gb: 512
+ ocs_dynamic_storage_class: managed-premium
+Warning
You are not allowed to specify the OCP version of the ARO cluster. The latest current version is provisioned automatically instead no matter what value is specified in the "ocp_version" parameter. The "ocp_version" parameter is mandatory for compatibility with other layers of the provisioning, such as the OpenShift client. For instance, the value is used by the process which downloads and installs the oc client. Please, specify the value according to what OCP version will be provisioned.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name of the OpenShift cluster | Yes | |
| azure_name | Name of the azure element in the configuration | Yes | |
| domain_name | Domain mame of the cluster, if you want to override the name generated by Azure | No | |
| ocp_version | The OpenShift version. If you want to install 4.10, specify "4.10" | Yes | >= 4.6 |
| cloud_native_toolkit | Must the Cloud Native Toolkit (OpenShift GitOps) be installed? | No | True, False (default) |
| oadp | Must the OpenShift Advanced Data Protection operator be installed | No | True, False (default) |
| network | Cluster network attributes | Yes | |
| network.pod_cidr | CIDR of pod network | Yes | Must be a minimum of /18 or larger. |
| network.service_cidr | CIDR of service network | Yes | Must be a minimum of /18 or larger. |
| openshift_logging[] | Logging attributes for OpenShift cluster, see OpenShift logging | No | |
| upstream_dns[] | Upstream DNS servers(s), see Upstream DNS Servers | No | |
| gpu | Control Node Feature Discovery and NVIDIA GPU operators | No | |
| gpu.install | Must Node Feature Discovery and NVIDIA GPU operators be installed (Once installed, False does not uninstall). auto will install the operators if needed by any of the Cloud Pak/watsonx | Yes | auto, True, False |
| openshift_ai | Control installation of OpenShift AI | No | |
| openshift_ai.install | Must OpenShift AI be installed (Once installed, False does not uninstall). auto will install OpenShift AI if needed by any of the Cloud Pak/watsonx components | Yes | auto, True, False |
| openshift_ai.channel | Which oeprator channel must be installed | No | auto (default), stable, … |
| mcg | Multicloud Object Gateway properties | No | |
| mcg.install | Must Multicloud Object Gateway be installed (Once installed, False does not uninstall) | Yes | True, False |
| mcg.storage_type | Type of storage supporting the object Noobaa object storage | Yes | storage-class |
| mcg.storage_class | Storage class supporting the Noobaa object storage | Yes | Existing storage class |
| openshift_storage[] | List of storage definitions to be defined on OpenShift, see below for further explanation | Yes |
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| openshift_storage[] | List of storage definitions to be defined on OpenShift | Yes | |
| storage_name | Name of the storage | Yes | |
| storage_type | Type of storage class to create in the OpenShift cluster | Yes | ocs or nfs |
| ocs_version | Version of the OCS operator. If not specified, this will default to the ocp_version | No | |
| ocs_storage_label | Label (or rather a name) to be used for the dedicated OCS nodes in the cluster - together with the combination of Azure location and zone id | Yes if storage_type is ocs | |
| ocs_storage_size_gb | Size of the OCS storage in Gibibytes (Gi) | Yes if storage_type is ocs | |
| ocs_dynamic_storage_class | Storage class that will be used for provisioning OCS. In Azure, you must select managed-premium | Yes if storage_type is ocs | managed-premium |
In cases where the OpenShift cluster is in an environment with limited internet connectivity, you may want OpenShift to pull Cloud Pak images from a private image registry (aka container registry). There may also be other reasons for choosing a private registry over the entitled registry.
The below steps outline how to configure a private registry for a Cloud Pak deployment. When the image_registry object is referenced by the Cloud Pak object (such as cp4d), the deployer makes the following changes in OpenShift so that images are pulled from the private registry:
image-registry-<name> and an entry for the registry is added to the global pull secret (secret pull-secret in project openshift-config).quay.io/opencloudio/zen-metastoredb@sha256:582cac2366dda8520730184dec2c430e51009a854ed9ccea07db9c3390e13b29 is mapped to registry.coc.uk.ibm.com:15000/opencloudio/zen-metastoredb@sha256:582cac2366dda8520730184dec2c430e51009a854ed9ccea07db9c3390e13b29.image.config.openshift.io/cluster. If a private registry with a self-signed certificate is configured, certificate authority's PEM secret must be created as a configmap in the openshift-config project. The deployer uses the vault secret referenced in registry_trusted_ca_secret property to create or update the configmap so that OpenShift can connect to the registry in a secure manner. Alternatively, you add the registry_insecure: true property to pull images without checking the certificate.image_registry🔗Defines a private registry that will be used for pulling the Cloud Pak container images from. Additionally, if the Cloud Pak entitlement key was specified at run time of the deployer, the images defined by the case files will be mirrored to this private registry.
image_registry:
+- name: cpd463
+ registry_host_name: registry.example.com
+ registry_port: 5000
+ registry_insecure: false
+ registry_trusted_ca_secret: cpd463-ca-bundle
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| name | Name by which the image registry is identified. | Yes | |
| registry_host_name | Host name or IP address of the registry server | Yes | |
| registry_port | Port that the image registry listens on. Default is the https port (443) | No | |
| registry_namespace | Namespace (path) within the registry that holds the Cloud Pak images. Mandatory only when using the IBM Cloud Container Registry (ICR) | No | |
| registry_insecure | Defines whether insecure registry access with a self-signed certificate is allowed | No | True, False (default) |
| registry_trusted_ca_secret | Defines the vault secret which holds the certificate authority bundle that must be used when connecting to this private registry. This parameter cannot be specified if registry_insecure is also specified. | No |
Warning
The registry_host_name you specify in the image_registry definition must also be available for DNS lookup within OpenShift. If the registry runs on a server that is not registered in the DNS, use its IP address instead of a host name.
When mirroring images, the deployer connects to the registry using the host name and port. If the port is omitted, the standard https protocol (443) is used. If a registry_namespace is specified, for example when using the IBM Container Registry on IBM Cloud, it will be appended to the registry URL.
The user and password to connect to the registry will be retrieved from the vault, using secret image-registry-<your_image_registry_name> and must be stored in the format registry_user:registry_password. For example, if you want to connect to the image registry cpd404 with user admin and password very_s3cret, you would create a secret as follows:
./cp-deploy.sh vault set \
+ -vs image-registry-cpd463 \
+ -vsv "admin:very_s3cret"
+If you need to connect to a private registry which is not signed by a public certificate authority, you have two choices: * Store the PEM certificate that that holds the CA bundle in a vault secret and specify that secret for the registry_trusted_ca_secret property. This is the recommended method for private registries. * Specify registry_insecure: false (not recommended): This means that the registry (and port) will be marked as insecure and OpenShift will pull images from it, even if its certificate is self-signed.
For example, if you have a file /tmp/ca.crt with the PEM certificate for the certificate authority, you can do the following:
./cp-deploy.sh vault set \
+ -vs cpd463-ca-bundle \
+ -vsf /tmp/ca.crt
+This will create a vault secret which the deployer will use to populate a configmap in the openshift-config project, which in turn is referenced by the image.config.openshift.io/cluster custom resource. For the above configuration, configmap cpd404-ca-bundle would be created and teh image.config.openshift.io/cluster would look something like this:
apiVersion: config.openshift.io/v1
+kind: Image
+metadata:
+...
+...
+ name: cluster
+spec:
+ additionalTrustedCA:
+ name: cpd463-ca-bundle
+If you want to use a private registry when running the deployer for a ROKS cluster on IBM Cloud, you must use the IBM Container Registry (ICR) service. The deployer will automatically create the specified namespace in the ICR and set up the credentials accordingly. Configure an image_registry object with the host name of the private registry and the namespace that holds the images. An example of using the ICR as a private registry:
image_registry:
+- name: cpd463
+ registry_host_name: de.icr.io
+ registry_namespace: cpd463
+The registry host name must end with icr.io and the registry namespace is mandatory. No other properties are needed; the deployer will retrieve them from IBM Cloud.
If you have already created the ICR namespace, create a vault secret for the image registry credentials:
./cp-deploy.sh vault set \
+ -vs image-registry-cpd463
+ -vsv "admin:very_s3cret"
+An example of configuring the private registry for a cp4d object is below:
cp4d:
+- project: cpd-instance
+ openshift_cluster_name: {{ env_id }}
+ cp4d_version: 4.8.3
+ image_registry_name: cpd463
+The Cloud Pak for Data installation refers to the cpd463 image_registry object.
If the ibm_cp_entitlement_key secret is in the vault at the time of running the deployer, the required images will be mirrored from the entitled registry to the private registry. If all images are already available in the private registry, just specify the --skip-mirror-images flag when you run the deployer.
Configure an image_registry object with the host name of the private registry and some optional properties such as port number, CA certificate and whether insecure access to the registry is allowed.
Example:
image_registry:
+- name: cpd463
+ registry_host_name: registry.example.com
+ registry_port: 5000
+ registry_insecure: false
+ registry_trusted_ca_secret: cpd463-ca-bundle
+Warning
The registry_host_name you specify in the image_registry definition must also be available for DNS lookup within OpenShift. If the registry runs on a server that is not registered in the DNS, use its IP address instead of a host name.
To create the vault secret for the image registry credentials:
./cp-deploy.sh vault set \
+ -vs image-registry-cpd463
+ -vsv "admin:very_s3cret"
+To create the vault secret for the CA bundle:
./cp-deploy.sh vault set \
+ -vs cpd463-ca-bundle
+ -vsf /tmp/ca.crt
+Where ca.crt looks something like this:
-----BEGIN CERTIFICATE-----
+MIIFszCCA5ugAwIBAgIUT02v9OdgdvjgQVslCuL0wwCVaE8wDQYJKoZIhvcNAQEL
+BQAwaTELMAkGA1UEBhMCVVMxETAPBgNVBAgMCE5ldyBZb3JrMQ8wDQYDVQQHDAZB
+cm1vbmsxFjAUBgNVBAoMDUlCTSBDbG91ZCBQYWsxHjAcBgNVBAMMFUlCTSBDbG91
+...
+mcutkgtbkq31XYZj0CiM451Qp8KnTx0=
+-----END CERTIFICATE-
+An example of configuring the private registry for a cp4d object is below:
cp4d:
+- project: cpd-instance
+ openshift_cluster_name: {{ env_id }}
+ cp4d_version: 4.8.3
+ image_registry_name: cpd463
+The Cloud Pak for Data installation refers to the cpd463 image_registry object.
If the ibm_cp_entitlement_key secret is in the vault at the time of running the deployer, the required images will be mirrored from the entitled registry to the private registry. If all images are already available in the private registry, just specify the --skip-mirror-images flag when you run the deployer.
You can configure Red Hat Single Sign-on (SSO) to be installed on the OpenShift cluster as an Identity Provider (IdP). Red Hat SSO implements the open-source Keycloak project which offers a user registry and can also federate other IdPs.
openshift_redhat_sso🔗An openshift_redhat_sso resource indicates that the Red Hat Single Sign-on operator must be installed on the referenced OpenShift cluster. A single SSO configuration can have only 1 Keycloak realms defined. The Keycloak realm holds all configuration needed for authentication. If you want to host more than 1 Keycloak server on the cluster, specify multiple openshift_redhat_sso entries, each with its own keycloak_name. The keycloak_name also determines the OpenShift project that will be created.
openshift_redhat_sso:
+- openshift_cluster_name: "{{ env_id }}"
+ keycloak_name: ibm-keycloak
+ groups:
+ - name: kc-cp4d-admins
+ state: present
+ - name: kc-cp4d-data-engineers
+ state: present
+ - name: kc-cp4d-data-scientists
+ state: present
+ - name: kc-cp4d-monitors
+ state: present
+The above configuration installs the Red Hat SSO operator in OpenShift project ibm-keycloak and creates a Keycloak instance named ibm-keycloak. The instance has a single realm: master which contains the groups, users and clients which are then leveraged by Cloud Pak Foundational Services.
Currently you can only define Keycloak groups which are later mapped to Cloud Pak for Data user groups. Creating users and setting up federated identity providers must be done by logging into Keycloak.
The Keycloak name is referenced in the Zen Access Control resource and this is also where the mapping from Keycloak groups to Cloud Pak for Data groups takes place.
| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| openshift_cluster_name | Name of OpenShift cluster onto which the Red Hat SSO operator is installed | Yes. if more than 1 openshift resource in the configuration | |
| keycloak_name | Name of the Keycloak server, this also determines the name of the project into which the Keycloak server will be created | Yes | |
| .groups[] | Groups that will be created in the Keycloak realm | Yes | |
| .name | Name of the Keycloak group | Yes | |
| .state | Whether the group is present or absent | Yes | present, absent |
Configuration of the topology to be deployed typically boils down to choosing the cloud infrastructure you want to deploy, then choosing the type of OpenShift and storage, integrating with infrastructure services and then setting up the Cloud Pak(s). For most initial implementations, a basic deployment will suffice and later this can be extended with additional configuration.
Depicted below is the basic deployment topology, followed by a topology with all bells and whistles.
For more details on each of the configuration elements, refer to:
For more details about extended deployment, refer to:
Throughout the deployment process, the Cloud Pak Deployer will create secrets in a vault and retrieve them later. Examples of secrets are: ssh keys, Cloud Pak for Data admin password. Additionally, when provisioning infrastructure no the IBM Cloud, the resulting Terraform state file is also stored in the vault so it can be used later if the configuration needs to be changed.
Configuration of the vault is done through a vault object in the configuration. If you want to use the file-based vault in the status directory, you do not need to configure anything.
The following Vault implementations can be used to store and retrieve secrets: - File Vault (no encryption) - IBM Cloud Secrets Manager - Hashicorp Vault (token authentication) - Hashicorp Vault (certificate authentication)
The File Vault is the default vault and also the simplest. It does not require a password and all secrets are stored in base-64 encoding in a properties file under the <status_directory>/vault directory. The name of the vault file is the environment_name you specified in the global configuration, inventory file or at the command line.
All of the other vault options require some secret manager (IBM Cloud service or Hashicorp Vault) to be available and you need to specify a password or provide a certificate.
Sample Vault config:
vault:
+ vault_type: file-vault
+ vault_authentication_type: none
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| vault_type | Chosen implementation of the vault | Yes | file-vault, ibmcloud-vault, hashicorp-vault |
file-vault🔗| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| vault_authentication_type | Authentication method for the file vault | No | none |
ibmcloud-vault🔗| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| vault_authentication_type | Authentication method for the file vault | No | api-key |
| vault_url | URL for the IBM Cloud secrets manager instance | Yes |
hashicorp-vault🔗| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| vault_authentication_type | Authentication method for the file vault | No | api-key, certificate |
| vault_url | URL for the Hashicorp vault, this is typically https://hostname:8200 | Yes | |
| vault_api_key | When authentication type is api-key, the field to authenticate with | Yes | |
| vault_secret_path | Default secret path to store and retrieve secrets into/from | Yes | |
| vault_secret_field | Default field to store or retrieve secrets | Yes | |
| vault_secret_path_append_group | Determines whether or not the secrete group will be appended to the path | Yes | True (default), False |
| vault_secret_base64 | Depicts if secrets are stored in base64 format for Hashicorp Vault | Yes | True (default), False |
This stage focuses on post-installation configuration of the Cloud Paks and cartridges.
When provisioning on IBM Cloud ROKS, a CA-signed certificate for the ingress subdomain is automatically generated in the IBM Cloud certificate manager. The deployer retrieves the certificate and adds it to the secret that stores the certificate key. This will avoid getting a warning when opening the Cloud Pak for Data home page.
For Cloud Pak for Data you can configure:
cp4d_saml_config object, the deployer configures the user management pods to redirect logins to the identity provider (idP) of choice.cp4d_ldap_config object so that the deployer configures it for Cloud Pak for Data. If SAML has been configured for authentication, the configured LDAP server is only used for access management.cp4d_user_group_configuration.Some cartridges such as Data Virtualization have the ability to create one or more instances to run an isolated installation of the cartridge. If instances have been configured for the cartridge, this steps provisions them. The following Cloud Pak for Data cartridges are currently supported for creating instances:
analytics-engine)db2)ca)dv)Cloud Pak for Data does not support group-defined access to cartridge instances. After creation of the instances (and also when the deployer is run with the --cp-config-only flag), the permissions of users accessing the instance is configured.
For Cognos Analytics, the Cognos Authorization process is run to apply user group permissions to the Cognos Analytics instance.
Cloud Pak for Data defines data source connections at the platform level and these can be reused in some cartridges like Watson Knowledge Catalog and Watson Studio. The cp4d_connection object defines each of the platform connections that must be managed by the deployer.
If you want to back up or restore platform connections, the cp4d_backup_restore_connections object defines the JSON file that will be used for backup and restore.
This stage focuses on the configuration of the provisioned infrastructure.
In a configuration scenario where NFS is used for OpenShift storage, the NFS server must be provisioned as a VSI within the VPC that contains the OpenShift cluster. It is best practice to shield off the NFS server from the outside world by using a jump host (bastion) to access it.
This steps configures the bastion host which has a public IP address to serve as a jump host to access other servers and services within the VPC.
Configures the NFS server using the specs in the nfs_server configuration object(s). It installs the required packages and sets up the NFSv4 service. Additionally, it will format the empty volume as xfs and export it so it can be used by the managed-nfs-storage storage class in the OpenShift cluster.
This steps takes care of configuring the storage classes in the OpenShift cluster. Storage classes are an abstraction of the underlying physical and virtual storage. When run, it processes the openshift_storage elements within the current openshift configuration object.
Two types of storage classes can be automatically created and configured:
Creates the managed-nfs-storage OpenShift storage class using the specified nfs_server_name which references an nfs_server configuration object.
Activates the ROKS cluster's OpenShift Container Storage add-on to install the operator into the cluster. Once finished with the preparation, the OcsCluster OpenShift object is created to provision the storage cluster. As the backing storage the ibmc-vpc-block-metro-10iops-tier storage class is used, which has the appropriate IO characteristics for the Cloud Paks.
Info
Both NFS and OCS storage classes can be created but only 1 storage class of each type can exist in the cluster at the moment. If more than one storage class of the same type is specified, the configuration will fail.
The automated cognos authorization capability uses LDAP groups to assign users to a Cognos Analytics Role, which allows these users to login to IBM Cloud Pak for Data and access the Cognos Analytics instance. This capability will perform the following tasks: - Create a User Group and assign the associated LDAP Group(s) and Cloud Pak for Data role(s) - For each member of the LDAP Group(s) part of the User Group, create the user as a Cloud Pak for Data User and assigned the Cloud Pak for Data role(s) - For each member of the LDAP Group(s) part of the User Group, assign membership to the Cognos Analytics instance and authorize for the Cognos Analytics Role
If the User Group is already present, validate all LDAP Group(s) are associated with the User Group. Add the LDAP Group(s) not yet assiciated to the User Group. Existing LDAP groups will not be removed from the User Group
If a User is already present in Cloud Pak for Data, it will not be updated.
If a user is already associated with the Cognos Analytics instance, keep its original membership and do not update the membership
Prior to running the script, ensure: - LDAP configuration in IBM Cloud Pak for Data is completed and validated - Cognos Analytics instance is provisioned and running in IBM Cloud Pak for Data - The role(s) that will be associated with the User Group are present in IBM Cloud Pak for Data
The script is available in automation-roles/50-install-cloud-pak/cp4d-service/files/assign_CA_authorization.sh.
Run the script without arguments to show its usage help.
# ./assign_CA_authorization.sh
+Usage:
+
+assign_CA_authorization.sh
+ <CLOUD_PAK_FOR_DATA_URL>
+ <CLOUD_PAK_FOR_DATA_LOGIN_USER>
+ <CLOUD_PAK_FOR_DATA_LOGIN_PASSWORD>
+ <CLOUD_PAK_FOR_DATA_USER_GROUP_NAME>
+ <CLOUD_PAK_FOR_DATA_USER_GROUP_DESCRIPTION>
+ <CLOUD_PAK_FOR_DATA_USER_GROUP_ROLES_ASSIGNMENT>
+ <CLOUD_PAK_FOR_DATA_USER_GROUP_LDAP_GROUPS_MAPPING>
+ <CLOUD_PAK_FOR_DATA_COGNOS_ANALYTICS_ROLE>
+Using the command example provided by the ./assign_CA_authorization.sh command, run the script with its arguments
# ./assign_CA_authorization.sh \
+ https://...... \
+ admin \
+ ******** \
+ "Cognos User Group" \
+ "Cognos User Group Description" \
+ "wkc_data_scientist_role;zen_administrator_role" \
+ "cn=ca_group,ou=groups,dc=ibm,dc=com" \
+ "Analytics Viewer"
+Validation
Confirm all required arguments are provided.
Confirm at least 1 User Group Role assignment is provided.
Confirm at least 1 LDAP Group is provided.
Login to Cloud Pak for Data and generate a Bearer token
Using the provided IBM Cloud for Data URL, username and password, login to Cloud pak for Data and generate the Bearer token used for subsequent commands. Exit with an error if the login to IBM Cloud Pak for Data fails.
Confirm the provided User Group role(s) are present in Cloud Pak for Data
Acquire all Cloud Pak for Data roles and confirm the provided User Group role(s) are one of the existing Cloud Pak for Data roles. Exit with an error if a role is provided which is not currently present in IBM Cloud Pak for Data.
Confirm the provided Cognos Analytics role is valid
Ensure the provided Cognos Analytics role is one of the available Cognos Analytics roles. Exit with an error if a Cognos Analytics role is provided that does not match with the available Cognos Analytics roles.
Confirm LDAP is configured in IBM Cloud Pak for Data
Ensures the LDAP configuration is completed. Exit with an error if there is no current LDAP configuration.
Confirm the provided LDAP groups are present in the LDAP User Registry
Using IBM Cloud Pak for Data, query whether the provided LDAP groups are present in the LDAP User registry. Exit with an error if a LDAP Group is not available.
Confirm if the IBM Cloud Pak for Data User Group exists
Queries the IBM Cloud Pak for Data User Groups. If the provided User Group exists, acquire the Group ID.
If the IBM Cloud Pak for Data User Group does not exist, create it
If the User Group does not exist, create it, and assign the IBM Cloud Pak for Data Roles and LDAP Groups to the new User Group
If the IBM Cloud Pak for Data User Group does exist, validate the associated LDAP Groups
If the User Group already exists, confirm all provided LDAP groups are associated with the User Group. Add LDAP groups that are not yet associated.
Get the Cognos Analytics instance ID
Queries the IBM Cloud Pak for Data service instances and acquires the Cognos Analytics instance ID. Exit with an error if no Cognos Analytics instance is available
Ensure each user member of the IBM Cloud Pak for Data User Group is an existing user
Each user that is member of the provided LDAP groups, ensure this member is an IBM Cloud Pak for Data User. Create a new user with the provided User Group role(s) if the the user is not yet available. Any existing User(s) will not be updated. If Users are removed from an LDAP Group, these users will not be removed from Cloud Pak for Data.
Ensure each user member of the IBM Cloud Pak for Data User Group is associated to the Cognos Analytics instance
Each user that is member of the provided LDAP groups, ensure this member is associated to the Cognos Analytics instance with the provided Cognos Analytics role. Any user that is already associated to the Cognos Analytics instance will have its Cognos Analytics role updated to the provided Cognos Analytics Role
For Cloud Pak for Data, this stage does the following:
cp4d_assetcp4d_monitors elements.See cp4d_asset for more details.
See cp4d_monitors for more details.
This stage focuses on preparing the OpenShift cluster for installing the Cloud Pak(s) and then proceeds with the installation of Cloud Paks and the cartridges. The below documentation will start with a list of steps that will be executed for all Cloud Paks, then proceed with Cloud Pak specific activities. The execution of the steps may slightly differ from the sequence in the documentation.
Sections:
Before going ahead with the mirroring of container images and installation of Cloud Pak for Data, the previous configuration (if any) is retrieved from the vault to determine if a Cloud Pak for Data instance has been removed. If a previously installed cp4d object no longer exists in the current configuration, its associated instance is removed from the OpenShift cluster.
First, the custom resources are removed from the OpenShift project. This happens with a grace period of 5 minutes. After the grace period has expired, OpenShift automatically forcefully deletes the custom resource and its associated definitions. Then, the control plane custom resource Ibmcpd is removed and finally the namespace (project). For the namespace deletion, a grace period of 10 minutes is applied.
When installing the Cloud Paks, images must be pulled from an image registry. All Cloud Paks support pulling images directly from the IBM Entitled Registry using the entitlement key, but there may be situations this is not possible, for example in air-gapped environents, or when images must be scanned for vulnerabilities before they are allowed to be used. In those cases, a private registry will have to be set up.
The Cloud Pak Deployer can mirror images to a private registry from the entitled registry. On IBM Cloud, the deployer is also capable of creating a namespace in the IBM Container Registry and mirror the images to that namespace.
When a private registry has been specified in the Cloud Pak entry (using the image_registry_name property), the necessary OpenShift configuration changes will also be made.
If OpenShift is deployed on IBM Cloud (ROKS), the IBM Container Registry should be used as the private registry from which the images will be pulled. Images in the ICR are organized by namespace and can be accessed using an API key issued for a service account. If an image_registry object is specified in the configuration, this process will take care of creating the service account, then the API key and it will store the API key in the vault.
If an image registry has been specified for the Cloud Pak using the image_registry_name property, the referenced image_registry entry is looked up in the configuration and the credentials are retrieved from the vault. Then the connection to the registry is tested by logging on.
Cloud Pak for Data requires a number of cluster-wide settings:
ImageContentSourcePolicy if images must be pulled from a private registryTuned object to set kernel semaphores and other properties of CoreOS containers being spun upFor all OpenShift clusters, except ROKS on IBM Cloud, these settings are applied using OpenShift configuration objects and then picked up by the Machine Config Operator. This operator will then apply the settings to the control plane and compute nodes as appropriate and reload them one by one.
To avoid having to reload the nodes more than once, the Machine Config Operator is paused before the settings are applied. After all setup, the Machine Config Operator is released and the deployment process will then wait until all nodes are ready with the configuration applied.
As mentioned before, ROKS on IBM Cloud does not include the Machine Config Operator and would normally require the compute nodes to be reloaded (classic ROKS) or replaced (ROKS on VPC) to make the changes effective. While implementing this process, we have experienced intermittent reliability issues where replacement of nodes never finished or the cluster ended up in a unusable state. To avoid this, the process is applying the settings in a different manner.
On every node, a cron job is created which starts every 5 minutes. It runs a script that checks if any of the cluster-wide settings must be (re-)applied, then updates the local system and restarts the crio and kubelet daemons. If no settings are to be adjusted, the daemons will not be restarted and therefore the cron job has minimal or no effect on the running applications.
Compute node changes that are made by the cron job: ImageContentSourcePolicy: File /etc/containers/registries.conf is updated to include registry mirrors for the private registry. Kubelet: File /etc/kubernetes/kubelet.conf is appended with the allowedUnsafeSysctls entries. CRI-O: pids_limit and default_ulimit changes are made to the /etc/crio/crio.conf file. Pull secret: The registry and credentials are appended to the /.docker/config.json configuration.
There are scenarios, especially on IBM Cloud Satellite, where custom changes must be applied to the compute nodes. This is possible by adding the apply-custom-node-settings.sh to the assets directory within the CONFIG_DIR directory. Once Kubelet, CRI-O and other changes have been applied, this script (if existing) is run to apply any additional configuration changes to the compute node.
By setting the NODE_UPDATED script variable to 1 you can tell the deployer to restart the crio and kubelet daemons.
WARNING: You should never set the NODE_UPDATED script variable to 0 as this will cause previous changes to the pull secret, ImageContentSourcePolicy and others not to become effective.
WARNING: Do not end the script with the exit command; this will stop the calling script from running and therefore not restart the daemons.
Sample script:
#!/bin/bash
+
+#
+# This is a sample script that will cause the crio and kubelet daemons to be restarted once by checking
+# file /tmp/apply-custom-node-settings-run. If the file doesn't exist, it creates it and sets NODE_UPDATED to 1.
+# The deployer will observe that the node has been updated and restart the daemons.
+#
+
+if [ ! -e /tmp/apply-custom-node-settings-run ];then
+ touch /tmp/apply-custom-node-settings-run
+ NODE_UPDATED=1
+fi
+If a private image registry is specified, and if the IBM Cloud Pak entitlement key is available in the vault (cp_entitlement_key secret), the Cloud Pak case files for the Foundational Services, the Cloud Pak control plane and cartridges are downloaded to a subdirectory of the status directory that was specified. Then all images defined for the cartridges are mirrored from the entitled registry to the private image registry. Dependent on network speed and how many cartridges have been configured, the mirroring can take a very long time (12+ hours). All images which have already been mirrored to the private registry are skipped by the mirroring process.
Even if all images have been mirrored, the act of checking existence and digest can still take a bit of time (10-15 minutes). To avoid this, you can remove the cp_entitlement_key secret from the vault and unset the CP_ENTITLEMENT_KEY environment variable before running the Cloud Pak Deployer.
The images of the operators which control the Cloud Pak are defined in OpenShift CatalogSource objects which reside in the openshift-marketplace project. Operator subscriptions subsequently reference the catalog source and define the update channel. When images are pulled from the entitled registry, most subscriptions reference the same ibm-operator-catalog catalog source (and also a Db2U catalog source). If images are pulled from a private registry, the control plane and also each cartridge reference their own catalog source in the openshift-marketplace project.
This step creates the necessary catalog sources, dependent on whether the entitled registry or a private registry is used. For the entitled registry, it creates the catalog source directly using a YAML template; when using a private registry, the cloudctl case command is used for the control plane and every cartridge to install the catalog sources and their dependencies.
Most custom resources defined by the cartridge operators require some back-end storage. To be able to reference the correct OpenShift storage classes, they are retrieved based on the openshift_storage_name property of the Cloud Pak object.
When using express install, the Cloud Pak for Data operator also installs the Cloud Pak Foundational Services. Consecutively, this part of the deployer:
When the Cloud Pak for Data operator has been installed, the process continues by creating an OperandRequest object for the platform operator which manages the project in the which Cloud Pak for Data instance is installed. Then it creates an Ibmcpd custom resource in the project which installs the controle plane with nginx the metastore, etc.
The Cloud Pak for Data control plane is a pre-requisite for all cartridges so at this stage, the deployer waits until the Ibmcpd status reached the Completed state.
Once the control plane has been installed successfully, the deployer generates a new strong 25-character password for the Cloud Pak for Data admin user and stores this into the vault. Additionally, the admin-user-details secret in the OpenShift project is updated with the new password.
Now that the control plane has been installed in the specified OpenShift project, cartridges can be installed. Every cartridge is controlled by its own operator subscription in the operators project and a custom resource. The deployer iterates twice over the specified cartridges, first to create the operator subscriptions, then to create the custom resources.
This steps creates subscription objects for each cartridge in the operators project, using a YAML template that is included in the deployer code and the subscription_channel specified in the cartridge definition. Keeping the subscription channel separate delivers flexibility when new subscription channels become available over time.
Once the subscription has been created, the deployer waits for the associate CSV(s) to be created and reach the Installed state.
If this is not the first installation, earlier configured cartridges may have been removed. This steps iterates over all supported cartridges and checks if the cartridge has been installed and wheter it exists in the configuration of the current cp4d object. If the cartridge is no longer defined, its custom resource is removed; the operator will then take care of removing all OpenShift configuration.
This steps creates the Custom Resources for each cartridge. This is the actual installation of the cartridge. Cartridges can be installed in parallel to a certain extent and the operator will wait for the dependencies to be installed first before starting the processes. For example, if Watson Studio and Watson Machine Learning are installed, both have a dependency on the Common Core Services (CCS) and will wait for the CCS object to reach the Completed state before proceeding with the install. Once that is the case, both WS and WML will run the installation process in parallel.
Installation of the cartridges can take a very long time; up to 5 hours for Watson Knowledge Catalog. While cartridges are being installed, the deployer checks the states of all cartridges on a regular basis and reports these in a log file. The deployer will retry until all specified cartridges have reached the Completed state.
If LDAP has been configured for the Cloud Pak for Data element, it will be configured after all cartridges have finished installing.
When running the Cloud Pak Deployer (cp-deploy env apply), a series of pre-defined stages are followed to arrive at the desired end-state.
In this stage, the following activities are executed:
In this stage, the following activities are executed:
config directorycp-deploy commanddefaults directoryIn this stage, the following activities are executed:
In this stage, the following activities are executed:
In this stage, the following activities are executed:
In this stage, the following activities are executed:
In this stage, the following activities are executed:
This stage mainly takes care of checking the configuration and expanding it where necessary so it can be used by subsequent stages. Additionally, the preparation also calls the roles that will generate Terraform or other configuration files which are needed for provisioning and configuration.
All yaml files in the config directory of the specified CONFIG_DIR are processed and a composite JSON object, all_config is created, which contains all configuration.
While processing the objects defined in the config directory files, the defaults directory is also processed to determine if any supplemental "default" variables must be added to the configuration objets. This makes it easy for example to ensure VSIs always use the correct Red Hat Enterprise Linux image available on IBM Cloud.
You will find the generator roles under the automation-generators directory. There are cloud-provider dependent roles such as openshift which have a structure dependent on the chosen cloud provider and there are generic roles such as cp4d which are not dependent on the cloud provider.
To find the appropriate role for the object, the generator first checks if the role is found under the specified cloud provider directory. If not found, it will call the role under generic.
Each of the objects have a syntax checking module called preprocessor.py. This Python program checks the attributes of the object in question and can also add defaults for properties which are missing. All errors found are collected and displayed at the end of the generator.
This stage will provision the infrastructure that was defined in the input configuration files. Currently, this has only been implemented for IBM Cloud.
The IBM Cloud infrastructure provisioning runs Terraform to initially provision the infrastructure components such as VPC, VSIs, security groups, ROKS cluster and others. Also, if changes have been made in the configuration, Terraform will attempt to make the changes to reach the desired end-state.
Based on the chosen action (apply or destroy), Terraform is instructed to provision or change the infrastructure components or to destroy everything.
The Terraform state file (tfstate) is maintained in the vault and is critical to enable dynamic updates to the infrastructure. If the state file is lost or corrupted, updates to the infrastructure will have to be done manually. The Ansible tasks have been built in a way that the Terraform state file is always persisted into the vault, even if the apply or destroy process has failed.
There are 3 main steps:
This step initializes the Terraform provider (ibm) with the correct version. If needed, the Terraform modules for the provider are downloaded or updated.
Applying changes to the infrastructure using Terraform based on the input configuration files may cause critical components to be replaced (destroyed and recreated). The plan step checks what will be changed. If infrastructure components are destroyed and the --confirm-destroy parameter has not be specified for the deployer, the process is aborted.
This is the execution of the plan and will provision new infrastructure (apply) or destroy everything (destroy).
While the Terraform apply or destroy process is running, a .tfstate file is updated on disk. When the command completes, the deployer writes this as a secret to the vault so it can be used next time to update (or destroy) the infrastructure components.
This is the final stage before returning control to the process that started the deployer. Here tests to check that the Cloud Pak and its cartridges has been deployed correctly and that everything is running as expected.
The method for smoke tests should be dynamic, for example by referencing a Git repository and context (directory within the repository); the code within that directory then deploys the asset(s).
This "smoke test" finds the route of the Cloud Pak for Data instance(s) and retrieves the admin password from the vault which is then displayed.
Example:
['CP4D URL: https://cpd-cpd.fke09-10-a939e0e6a37f1ce85dbfddbb7ab97418-0000.eu-gb.containers.appdomain.cloud', 'CP4D admin password: ITnotgXcMTcGliiPvVLwApmsV']
+With this information you can go to the Cloud Pak for Data URL and login using the admin user.
In this stage, the following activities are executed:
| Phase | Step | Time in minutes | Comments |
|---|---|---|---|
| 10 - Validation | 3 | ||
| 20 - Prepare | Generators | 3 | |
| 30 - Provision infrastructure | Create VPC | 1 | |
| Create VSI without storage | 5 | ||
| Create VSI with storage | 10 | ||
| Create VPC ROKS cluster | 45 | ||
| Install ROKS OCS add-on and create storage classes | 45 | ||
| 40 - Configure infrastructure | Install NFS on VSIs | 10 | |
| Create NFS storage classes | 5 | ||
| Create private container registry namespace | 5 | ||
| 50 - Install Cloud Pak | Prepare OpenShift for Cloud Pak for Data install | 60 | During this step, the compute nodes may be replaced and also the Kubernetes services may be restarted. |
| Mirror Cloud Pak for Data images to private registry (only done when using private registry) | 30-600 | If the entitled registry is used, this step will be skipped. When using a private registry, if images have already been mirrored, the duration will be much shorter, approximately 10 minutes. | |
| Install Cloud Pak for Data control plane | 20 | ||
| Create Cloud Pak for Data subscriptions for cartridges | 15 | ||
| Install cartridges | 20-300 | The amount of time really depends on the cartridges being installed. In the table below you will find an estimate of the installation time for each cartridge. Cartridges will be installed in parallel through the operators. | |
| 60 - Configure Cloud Pak | Configure Cloud Pak for Data LDAP | 5 | |
| Provision instances for cartridges | 30-60 | For cartridges that have instances defined. Creation of the instances will run in parallel where possible. | |
| Configure cartridge and instance permissions based on LDAP config | 10 | ||
| 70 - Deploy assets | No activities yet | 0 | |
| 80 - Smoke tests | Show Cloud Pak for Data cluster details | 1 |
| Cartridge | Full name | Installation time | Instance provisioning time | Dependencies |
|---|---|---|---|---|
| cpd_platform | Cloud Pak for Data control plane | 20 | N/A | |
| ccs | Common Core Services | 75 | N/A | |
| db2aas | Db2 as a Service | 30 | N/A | |
| iis | Information Server | 60 | N/A | ccs, db2aas |
| ca | Cognos Analytics | 20 | 45 | ccs |
| planning-analytics | Planning Analytics | 15 | N/A | |
| watson_assistant | Watson Assistant | 70 | N/A | |
| watson-discovery | Watson Discovery | 100 | N/A | |
| ] watson-ks | Watson Knowledge Studio | 20 | N/A | |
| watson-speech | Watson Speech to Text and Text to Speech | 20 | N/A | |
| wkc | Watson Knowledge Catalog | 90 | N/A | ccs, db2aas, iis |
| wml | Watson Machine Learning | 45 | N/A | ccs |
| ws | Watson Studio | 30 | N/A | ccs |
Examples:
For convenience, the Cloud Pak Deployer includes a script that removes the Cloud Pak for Data instance from the OpenShift cluster, then Cloud Pak Foundational Services and finally the catalog sources and CRDs.
Steps:
./scripts/cp4d/cp4d-delete-instance.sh <CP4D_project>You will have to confirm that you want to delete the instance and all other artifacts.
Warning
Please be very careful with this command. Ensure you are connected to the correct OpenShift cluster and that no other Cloud Paks use operator namespace. The action cannot be undone.
When choosing the "simple" sample configuration for ROKS VPC on IBM Cloud, the deployer also provisions a Virtual Server Instance and installs a standard NFS server on it. In some cases you may want to get access to the NFS server for troubleshooting.
For security reasons, the NFS server can only be reached via a bastion server that is connected to the internet, i.e. use the bastion server as a jump host, this to avoid exposing NFS volumes to the outside world and provide an extra layer of protection. Additionally, password login is disabled on both the bastion and NFS servers and one must use the private SSH key to connect.
Getting SSH access to the NFS server is easiest from within the deployer container as it has all tools installed to extract the IP addresses from the Terraform state file.
Optional: Ensure that the environment variables for the configuration and status directories are set. If not specified, the directories are assumed to be $HOME/cpd-config and $HOME/cpd-status.
export STATUS_DIR=$HOME/cpd-status
+export CONFIG_DIR=$HOME/cpd-config
+Start the deployer command line.
./cp-deploy.sh env command
+-------------------------------------------------------------------------------
+Entering Cloud Pak Deployer command line in a container.
+Use the "exit" command to leave the container and return to the hosting server.
+-------------------------------------------------------------------------------
+Installing OpenShift client
+Current OpenShift context: pluto-01
+Access to both the bastion and NFS servers are typically protected by the same SSH key, which is stored in the vault. To list all vault secrets, run the command below.
cd /cloud-pak-deployer
+./cp-deploy.sh vault list
+./cp-deploy.sh vault list
+
+Starting Automation script...
+
+PLAY [Secrets] *****************************************************************
+Secret list for group sample:
+- ibm_cp_entitlement_key
+- sample-terraform-tfstate
+- cp4d_admin_zen_40_fke34d
+- sample-all-config
+- pluto-01-provision-ssh-key
+- pluto-01-provision-ssh-pub-key
+
+PLAY RECAP *********************************************************************
+localhost : ok=11 changed=0 unreachable=0 failed=0 skipped=21 rescued=0 ignored=0
+Then, retrieve the private key (in the above example pluto-01-provision-ssh-key) to an output file in your ~/.ssh directory, make sure it has the correct private key format (new line at the end) and permissions (600).
SSH_FILE=~/.ssh/pluto-01-rsa
+mkdir -p ~/.ssh
+chmod 600 ~/.ssh
+./cp-deploy.sh vault get -vs pluto-01-provision-ssh-key \
+ -vsf $SSH_FILE
+echo -e "\n" >> $SSH_FILE
+chmod 600 $SSH_FILE
+To connect to the NFS server, you need the public IP address of the bastion server and the private IP address of the NFS server. Obviously these can be retrieved from the IBM Cloud resource list (https://cloud.ibm.com/resources), but they are also kept in the Terraform "tfstate" file
./cp-deploy.sh vault get -vs sample-terraform-tfstate \
+ -vsf /tmp/sample-terraform-tfstate
+The below commands do not provide the prettiest output but you should be able to extract the IP addresses from them.
For the bastion node public (floating) IP address:
cat /tmp/sample-terraform-tfstate | jq -r '.resources[]' | grep -A 10 -E "ibm_is_float"
+ "type": "ibm_is_floating_ip",
+ "name": "pluto_01_bastion",
+ "provider": "provider[\"registry.terraform.io/ibm-cloud/ibm\"]",
+ "instances": [
+ {
+ "schema_version": 0,
+ "attributes": {
+ "address": "149.81.215.172",
+...
+ "name": "pluto-01-bastion",
+For the NFS server:
cat /tmp/sample-terraform-tfstate | jq -r '.resources[]' | grep -A 10 -E "ibm_is_instance|primary_network_interface"
+...
+--
+ "type": "ibm_is_instance",
+ "name": "pluto_01_nfs",
+ "provider": "provider[\"registry.terraform.io/ibm-cloud/ibm\"]",
+ "instances": [
+...
+--
+ "primary_network_interface": [
+...
+ "name": "pluto-01-nfs-nic",
+ "port_speed": 0,
+ "primary_ipv4_address": "10.227.0.138",
+In the above examples, the IP addresses are:
149.81.215.17210.227.0.138Finally, to get command line access to the NFS server:
BASTION_IP=149.81.215.172
+NFS_IP=10.227.0.138
+ssh -i $SSH_FILE \
+ -o ProxyCommand="ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
+ -i $SSH_FILE -W %h:%p -q $BASTION_IP" \
+ root@$NFS_IP
+Once you've finished exploring the NFS server, you can exit from it:
exit
+Finally, exit from the deployer container which is then terminated.
exit
+The Cloud Pak Deployer includes several samples which you can use to build your own configuration. You can find sample configuration yaml files in the sub-directories of the sample-configurations directory of the repository. Descriptions and topologies are also included in the sub-directories.
Warning
Do not make changes to the sample configurations in the cloud-pak-deployer directory, but rather copy it to your own home directory or somewhere else and then make changes. If you store your own configuration under the repository's clone, you may not be able to update (pull) the repository with changes applied on GitHub, or accidentally overwrite it.
Warning
The deployer expects to manage all objects referenced in the configuration files, including the referenced OpenShift cluster and Cloud Pak installation. If you have already pre-provisioned the OpenShift cluster, choose a configuration with existing-ocp cloud platform. If the Cloud Pak has already been installed, unexpected and undesired activities may happen. The deployer has not been designed to alter a pre-provisioned OpenShift cluster or existing Cloud Pak installation.
mkdir -p $HOME/cpd-config/config
+cp -r ./sample-configurations/roks-ocs-cp4d/config/* $HOME/cpd-config/config/
+cd $HOME/cpd-config/config
+installed. Additionally you can accept the Cloud Pak license in the config file by specifying accept_licenses: True. nano ./config/cp4d-450.yaml
+The configuration typically works without any configuration changes and will create all referenced objects, including the Virtual Private Cloud, subnets, SSH keys, ROKS cluster and OCS storage ndoes. There is typically no need to change address prefixes and subnets. The IP addresses used by the provisioned components are private to the VPC and are not externally exposed.
mkdir -p $HOME/cpd-config/config
+samples-configuration directory to the config directory, for example: cp ./sample-configurations/sample-dynamic/config-samples/ocp-ibm-cloud-roks-ocs.yaml $HOME/cpd-config/config/
+Copy the relevant "cp4d-…" file from the samples-configuration directory to the config directory, for example:
cp ./sample-configurations/sample-dynamic/config-samples/cp4d-462.yaml $HOME/cpd-config/config/
+Edit the "$HOME/cpd-config/config/cp4d-....yaml" file and select the cartridges to be installed by changing the state to installed. Additionally you can accept the Cloud Pak license in the config file by specifying accept_licenses: True.
nano $HOME/cpd-config/config/cp4d-463.yaml
+For more advanced configuration topics such as using a private registry, setting up transit gateways between VPCs, etc, go to the Advanced configuration section
Every configuration has a fixed directory structure, consisting of mandatory and optional subdirectories. 
Mandatory subdirectories:
config: Keeps one or more yaml files with your OpenShift and Cloud Pak configurationAdditionally, there are 3 optional subdirectories:
defaults: Directory that keeps the defaults which will be merged with your configurationinventory: Keep global settings for the configuration such as environment name or other variables used in the configsassets: Keeps directories of assets which must be deployed onto the Cloud Pakconfig directory🔗You can choose to keep only a single file per subdirectory or, for more complex configurations, you can create multiple yaml files. You can find a full list of all supported object types here: Configuration objects. The generator automatically merges all .yaml files in the config and defaults directory. Files with different extensions are ignored. In the sample configurations we split configuration of the OpenShift ocp-... and Cloud Pak cp4.-... objects.
For example, your config directory could hold the following files:
cp4d-463.yaml
+ocp-ibm-cloud-roks-ocs.yaml
+This will provision a ROKS cluster on IBM Cloud with OpenShift Data Foundation (fka OCS) and Cloud Pak for Data 4.0.8.
defaults directory (optional)🔗Holds the defaults for all object types. If a certain object property has not been specified in the config directory, it will be retrieved from the defaults directory using the flavour specified in the configured object. If no flavour has been selected, the default flavour will be chosen.
You should not need this subdirectory in most circumstances.
assets directory (optional)🔗Optional directory holding the assets you wish to deploy for the Cloud Pak. More information about Cloud Pak for Data assets which can be deployed can be found in object definition cp4d_asset. The directory can be named differently as well, for example cp4d-assets or customer-churn-demo.
inventory directory (optional)🔗The Cloud Pak Deployer pipeline has been built using Ansible and it can be configured using "inventory" files. Inventory files allow you to specify global variables used throughout Ansible playbooks. In the current version of the Cloud Pak Deployer, the inventory directory has become fully optional as the global_config and vault objects have taken over its role. However, if there are certain global variables such as env_id you want to pass via an inventory file, you can also do this.
User passwords, certificates and other "secret" information is kept in the vault, which can be either a flat file (not encrypted), HashiCorp Vault or the IBM Cloud Secrets Manager service. Some of the deployment configurations require that the vault is pre-populated with secrets which as needed during the deployment. For example, a vSphere deployment needs the vSphere user and password to authenticate to vSphere and Cloud Pak for Data SAML configuration requires the idP certificate
All samples default to the File Vault, meaning that the vault will be kept in the vault directory under the status directory you specify when you run the deployer. Detailed descriptions of the vault settings can be found in the sample inventory file and also here: vault settings.
Optional: Ensure that the environment variables for the configuration and status directories are set. If not specified, the directories are assumed to be $HOME/cpd-config and $HOME/cpd-status.
export STATUS_DIR=$HOME/cpd-status
+export CONFIG_DIR=$HOME/cpd-config
+Set vSphere user secret:
./cp-deploy.sh vault set \
+ --vault-secret vsphere-user \
+ --vault-secret-value super_user@vsphere.local
+Or, if you want to create the secret from an input file:
./cp-deploy.sh vault set \
+ --vault-secret kubeconfig \
+ --vault-secret-file ~/.kube/config
+If the configuration is kept in a GitHub repository, you can set environment variables to have the deployer pull the GitHub repository to the current server before starting the process.
Set environment variables.
export CPD_CONFIG_GIT_REPO="https://github.com/IBM/cloud-pak-deployer-config.git"
+export CPD_CONFIG_GIT_REF="main"
+export CPD_CONFIG_GIT_CONTEXT=""
+CPD_CONFIG_GIT_REPO: The clone URL of the GitHub repository that holds the configuration.CPD_CONFIG_GIT_REF: The branch, tag or commit ID to be cloned. If not specified, the repository's default branch will be cloned.CPD_CONFIG_GIT_CONTEXT: The directory within the GitHub repository that holds the configuration. This directory must contain the config directory under which the YAML files are kept.Info
When specifying a GitHub repository, the contents will be copied under $STATUS_DIR/cpd-config and this directory is then set as the configuration directory.
In some situations you may want to use a single configuration for deployment in different environments, such as development, acceptance test and production. The Cloud Pak Deployer uses the Jinja2 templating engine which is included in Ansible to pre-process the configuration. This allows you to dynamically adjust the configuration based on extra variables you specify at the command line.
Example:
./cp-deploy.sh env apply \
+ -e ibm_cloud_region=eu_gb \
+ -e env_id=jupiter-03 [--accept-all-liceneses]
+This passes the env_id and ibm_cloud_region variables to the Cloud Pak Deployer, which can then populate variables in the configuration. In the sample configurations, the env_id is used to specify the name of the VPC, ROKS cluster and others and overrides the value specified in the global_config definition. The ibm_cloud_region overrides region specified in the inventory file.
...
+vpc:
+- name: "{{ env_id }}"
+ allow_inbound: ['ssh']
+
+address_prefix:
+### Prefixes for the client environment
+- name: "{{ env_id }}-zone-1"
+ vpc: "{{ env_id }}"
+ zone: {{ ibm_cloud_region }}-1
+ cidr: 10.231.0.0/26
+...
+When running with the above cp-deploy.sh command, the snippet would be generated as:
...
+vpc:
+- name: "jupiter-03"
+ allow_inbound: ['ssh']
+
+address_prefix:
+### Prefixes for the client environment
+- name: "jupiter-03-zone-1"
+ vpc: "jupiter-03"
+ zone: eu-de-1
+ cidr: 10.231.0.0/26
+...
+The ibm_cloud_region variable is specified in the inventory file. This is another method of specifying variables for dynamic configuration.
You can even include more complex constructs for dynamic configuration, with if statements, for loops and others.
An example where the OpenShift OCS storage classes would only be generated for a specific environment (pluto-prod) would be:
openshift_storage:
+ - storage_name: nfs-storage
+ storage_type: nfs
+ nfs_server_name: "{{ env_id }}-nfs"
+{% if env_id == 'jupiter-prod' %}
+ - storage_name: ocs-storage
+ storage_type: ocs
+ ocs_storage_label: ocs
+ ocs_storage_size_gb: 500
+{% endif %}
+For a more comprehensive overview of Jinja2 templating, see https://docs.ansible.com/ansible/latest/user_guide/playbooks_templating.html
Warning
In most scenarios you will not need this type of configuration.
Alternative repositories and registries are mainly geared towards pre-GA use of the Cloud Paks where CASE files are downloaded from internal repositories and staging container image registries need to be used as images have not been released yet.
By default the Cloud Pak Deployer image is built on top of the olm-utils images in icr.io. If you're working with a pre-release of the Cloud Pak OLM utils image, you can override the setting as follows:
export CPD_OLM_UTILS_V2_IMAGE=cp.staging.acme.com:4.8.0
+Or, for Cloud Pak for Data 5.0:
export CPD_OLM_UTILS_V3_IMAGE=cp.staging.acme.com:5.0.0
+Subsequently, run the install commmand:
./cp-deploy.sh build
+When specifying a cp_alt_repo object in a YAML file, this is used for all Cloud Paks. The object triggers the following steps: * The following files are created in the /tmp/work directory in the container: play_env.sh, resolvers.yaml and resolvers_auth. * When downloading CASE files using the ibm-pak plug-in, the play_env sets the locations of the resolvers and authorization files. * Also, the locations of the case files for the Cloud Pak, Foundational Servides and Open Content are set in an enviroment variable. * Registry mirrors are configured using an ImageContentSourcePolicy resource in the OpenShift cluster. * Registry credentials are added to the OpenShift cluster's global pull secret.
The cp_alt_repo is configured like this:
cp_alt_repo:
+ repo:
+ token_secret: github-internal-repo
+ cp_path: https://raw.internal-repo.acme.com/cpd-case-repo/4.8.0/promoted/case-repo-promoted
+ fs_path: https://raw.internal-repo.acme.com/cloud-pak-case-repo/main/repo/case
+ opencontent_path: https://raw.internal-repo.acme.com/cloud-pak-case-repo/main/repo/case
+ registry_pull_secrets:
+ - registry: cp.staging.acme.com
+ pull_secret: cp-staging
+ - registry: fs.staging.acme.com
+ pull_secret: cp-fs-staging
+ registry_mirrors:
+ - source: cp.icr.com/cp
+ mirrors:
+ - cp.staging.acme.com/cp
+ - source: cp.icr.io/cp/cpd
+ mirrors:
+ - cp.staging.acme.com/cp/cpd
+ - source: icr.io/cpopen
+ mirrors:
+ - fs.staging.acme.com/cp
+ - source: icr.io/cpopen/cpfs
+ mirrors:
+ - fs.staging.acme.com/cp
+| Property | Description | Mandatory | Allowed values |
|---|---|---|---|
| repo | Repositories to be accessed and the Git token | Yes | |
| repo.token_secret | Secret in the vault that holds the Git login token | Yes | |
| repo.cp_path | Repository path where to find Cloud Pak CASE files | Yes | |
| repo.fs)path | Repository path where to find the Foundational Services CASE files | Yes | |
| repo.opencontent_path | Repository path where to find the Open Content CASE files | Yes | |
| registry_pull_secrets | List of registries and their pull secrets, will be used to configure global pull secret | Yes | |
| .registry | Registry host name | Yes | |
| .pull_secret | Vault secret that holds the pull secret (user:password) for the registry | Yes | |
| registry_mirrors | List of registries and their mirrors, will be used to configure the ImageContentSourcePolicy | Yes | |
| .source | Registry and path referenced by the Cloud Pak/FS pod | Yes | |
| .mirrors: | List of alternate registry locations for this source | Yes |
Before running the deployer with a cp_alt_repo object, you need to ensure the referenced secrets are present in the vault.
For the GitHub token, you need to set the token (typically a deploy key) to login to GitHub or GitHub Enterprise.
./cp-deploy.sh vault set -vs github-internal-repo=abc123def456
+For the registry credentials, specify the user and password separated by a colon (:):
./cp-deploy.sh vault set -vs cp-staging="cp-staging-user:cp-staging-password"
+You can also set these tokens on the cp-deploy.sh env apply command line.
./cp-deploy.sh env apply -f -vs github-internal-repo=abc123def456 -vs cp-staging="cp-staging-user:cp-staging-password
+To run the deployer you can now use the standard process:
./cp-deploy.sh env apply -v
+Cloud Pak Deployer automatically applies cluster and node settings before installing the Cloud Pak(s). Sometimes you may also want to automate applying these node settings without installing the Cloud Pak. For convenience, the repository includes a script that makes the same changes normally done through automation: scripts/cp4d/cp4d-apply-non-mco-cluster-settings.sh.
To apply the node settings, do the following:
CP_ENTITLEMENT_KEY environment variableCPD_PRIVATE_REGISTRY and CPD_PRIVATE_REGISTRY_CREDS environment variablesscripts/cp4d/cp4d-apply-non-mco-cluster-settings.sh script.The CPD_PRIVATE_REGISTRY value must reference the registry host name and optionally the port and namespace that must prefix the images. For example, if the images are kept in https://de.icr.io/cp4d-470, you must specify de.icr.io/cp4d-470 for the CPD_PRIVATE_REGISTRY environment variable. If images are kept in https://cust-reg:5000, you must specify cust-reg:5000 for the CPD_PRIVATE_REGISTRY environment variable.
For the CPD_PRIVATE_REGISTRY_CREDS value, specify both the user and password in a single string, separated by a colon (:). For example: admin:secret_passw0rd.
Warning
When setting the private registry and its credentials, the script automatically creates the configuration that will set up ImageContentSourcePolicy and global pull secret alternatives. This change cannot be undone using the script. It is not possible to set the private registry and later change to entitled registry. Changing the private registry's credentials can be done by re-running the script with the new credentials.
export CPD_PRIVATE_REGISTRY=de.icr.io/cp4d-470
+export CPD_PRIVATE_REGISTRY_CREDS="iamapikey:U97KLPYF663AE4XAQL0"
+./scripts/cp4d/cp4d-apply-non-mco-cluster-settings.sh
+Creating ConfigMaps and secret
+configmap "cloud-pak-node-fix-scripts" deleted
+configmap/cloud-pak-node-fix-scripts created
+configmap "cloud-pak-node-fix-config" deleted
+configmap/cloud-pak-node-fix-config created
+secret "cloud-pak-node-fix-secrets" deleted
+secret/cloud-pak-node-fix-secrets created
+Setting global pull secret
+/tmp/.dockerconfigjson
+info: pull-secret was not changed
+secret/cloud-pak-node-fix-secrets data updated
+Private registry specified, creating ImageContentSourcePolicy for registry de.icr.io/cp4d-470
+Generating Tuned config
+tuned.tuned.openshift.io/cp4d-ipc unchanged
+Writing fix scripts to config map
+configmap/cloud-pak-node-fix-scripts data updated
+configmap/cloud-pak-node-fix-scripts data updated
+configmap/cloud-pak-node-fix-scripts data updated
+configmap/cloud-pak-node-fix-scripts data updated
+Creating service account for DaemonSet
+serviceaccount/cloud-pak-crontab-sa unchanged
+clusterrole.rbac.authorization.k8s.io/system:openshift:scc:privileged added: "cloud-pak-crontab-sa"
+Recreate DaemonSet
+daemonset.apps "cloud-pak-crontab-ds" deleted
+daemonset.apps/cloud-pak-crontab-ds created
+Showing running DaemonSet pods
+NAME READY STATUS RESTARTS AGE
+cloud-pak-crontab-ds-b92f9 0/1 Terminating 0 12m
+cloud-pak-crontab-ds-f85lf 0/1 ContainerCreating 0 0s
+cloud-pak-crontab-ds-jlbvm 0/1 ContainerCreating 0 0s
+cloud-pak-crontab-ds-rbj65 1/1 Terminating 0 12m
+cloud-pak-crontab-ds-vckrs 0/1 ContainerCreating 0 0s
+cloud-pak-crontab-ds-x288p 1/1 Terminating 0 12m
+Waiting for 5 seconds for pods to start
+
+Showing running DaemonSet pods
+NAME READY STATUS RESTARTS AGE
+cloud-pak-crontab-ds-f85lf 1/1 Running 0 5s
+cloud-pak-crontab-ds-jlbvm 1/1 Running 0 5s
+cloud-pak-crontab-ds-vckrs 1/1 Running 0 5s
+For IBM Employees only
If you want to install a Cloud Pak for Data release that has not GA'ed yet, you can request access to the internal CASE repository and container image registries.
Refer to: https://github.ibm.com/CloudPakDeployer/cloud-pak-deployer-internal/blob/main/README.md, for configuration instructions.
The process of supporting multiple products, releases and patch levels within a release has great similarity to the git-flow model, which has been really well-described by Vincent Driessen in his blog post: https://nvie.com/posts/a-successful-git-branching-model/. This model has been and is still very popular with many software-development teams.
Below is a description of how a git-flow could be implemented with the Cloud Pak Deployer. The following steps are covered:
.
There are 4 Cloud Pak environments within the company's domain: Dev, UAT, Pre-prod and Prod. Each of these environments have a namespace in the company's registry (or an isolated registry could be created per environment) and the Cloud Pak release installed is represented by manifests in a branch of the Git repository, respectively dev, uat, pp and prod.
Organizing registries by namespace has the advantage that duplication of images can be avoided. Each of the namespaces can have their own set of images that have been approved for running in the associated environment. The image itself is referenced by digest (i.e., checksum) and organized on disk as such. If one tries to copy an image to a different namespace within the same registry, only a new entry is created, the image itself is not duplicated because it already exists.
The manifests (CASE files) representing the Cloud Pak components are present in each of the branches of the Git repository, or there is a configuration file that references the location of the case file, including the exact version number.
In the Cloud Pak Deployer, we have chosen to reference the CASE versions in the configuration, for example:
cp4d:
+- project: cpd-instance
+ openshift_cluster_name: {{ env_id }}
+ cp4d_version: 4.8.3
+ openshift_storage_name: ocs-storage
+ cartridges:
+ - name: cpfs
+ - name: cpd_platform
+ - name: ws
+ state: installed
+ - name: wml
+ size: small
+ state: installed
+If Cloud Pak for Data has been configured with a private registry in the deployer config, the deployer will mirror images from the IBM entitled registry to the private registry. In the above configuration, no private registry has been specified. The deployer will automatically download and use the CASE files to create the catalog sources.
With the initial status in place, the continuous adoption process may commence, using the principles of git-flow.
Git-flow addresses a couple of needs for continuous adoption:
The Git repository consists of 4 branches: dev, uat, pp and prd. At the start, release 4.0.0 is being implemented and it will go through the stages from dev to prd. When the installation has been tested in development, a pull request (PR) is done to promote to the uat branch. The PR is reviewed, and changes are then merged into the uat branch. After testing in the uat branch, the steps are repeated until the 4.0.0 release is eventually in production.
With each of the implementation and promotion steps, the registry namespaces and associated with the particular branch are updated with the images described in the manifests kept in the Git repository. Additionally, the changes are installed in the respective environments. The details of these processes will be outlined later.
New patches are received, committed and installed on the dev branch on a regular basis and when no issues are found, the changes are gathered into a PR for uat. When no issues are found for 2 weeks, another PR is done for the pp branch and eventually for prd. During this promotion flow, new patches are still being received in dev.
While version 4.0.2 is running in production, a critical defect is found for which a hot fix is developed. The hot fix is first committed to the pp branch and tested and then a PR is made to promote it to the prd branch. In the meantime, the dev and uat branches continue with their own release schedule. The hot fix is included in 4.0.4 which will be promoted as part of the 4.0.5 release.
The uat, pp and prd branches can be protected by a branch protection rule so that changes from dev can only be promoted (via a pull request) after an approving review or, when the intention is to promote changes in a fully automated manner, after passing status checks and testing. Read Managing a branch protection rule for putting in these controls in GitHub or Protected branches for GitLab.
With this flow, there is control over patches, promotion approvals and releases installed in each of the environments. Additional branches could be introduced if additional environments are in play or if different releases are being managed using the git-flow.
As discussed above, patches are first "developed" in the dev branch, i.e., changes are fed into the Git repository, images are loaded into the company's registry (dev namespace) and the installed into the Dev environment.
The process of receiving and installing the patches is common for all Cloud Paks: the cloudctl case tool downloads the CASE file associated with the operator version and the same CASE file can be used to upload images into the company's registry. Then a Catalog Source is created which makes the images available to the operator subscriptions, which in turn manage the various custom resources in the Cloud Pak instance. For example, the ws operator manages the Ws custom resource and this CR ensures that OpenShift deployments, secrets, Config Maps, Stateful Sets, and so forth are managed within the Cloud Pak for Data instance project.
In the git-flow example, Watson Studio release 4.0.2 is installed by updating the Catalog Source. Detailed installation steps for Cloud Pak for Data can be found in the IBM documentation.
Now that the hard work of managing changes to the Git repository branches and image registry namespaces has been done, we can look at the (automatic) deployment of the changes.
In a continuous adoption workflow, the implementation of new releases and patches is automated by means of a pipeline, which allows for deployment and testing in a predictable and controlled manner. A pipeline executes a series of steps to inspect the change and then run the command to install it in the respective environment. Moreover, after installation tests can be automatically executed. The most-popular tools for pipelines are ArgoCD, GitLab pipelines and Tekton (serverless).
To link the execution of a pipeline with the git-flow pull request, one can use ArcoCD or a GitHub/GitLab webhook. As soon as a PR is accepted and changes are applied to the Git branch, the pipeline is triggered and will run the Cloud Pak Deployer to automatically apply the changes according to the latest version.
When building or running the deployer in an environment with strict policies for internet access, you may have to specify the list of URLs that need to be accessed by the deployer.
| Location | Used for |
|---|---|
| registry.access.redhat.com | Base image |
| icr.io | olm-utils base image |
| cdn.redhat.com | Installing operating system packages |
| cdn-ubi.redhat.com | Installing operating system packages |
| rpm.releases.hashicorp.com | Hashicorp Vault integration |
| dl.fedoraproject.org | Extra Packages for Enterprise Linux (EPEL) |
| mirrors.fedoraproject.org | EPEL mirror site |
| fedora.mirrorservice.org | EPEL mirror site |
| pypi.org | Python packages for deployer |
| galaxy.ansible.com | Ansible Galaxy packages |
| Location | Used for |
|---|---|
| github.com | Case files, Cloud Pak clients: cloudctl, cpd-cli, cpdctl |
| gcr.io | Google Container Registry (GCR) |
| objects.githubusercontent.com | Binary content for github.com |
| raw.githubusercontent.com | Binary content for github.com |
| mirror.openshift.com | OpenShift client |
| ocsp.digicert.com | Certificate checking |
| subscription.rhsm.redhat.com | OpenShift subscriptions |
Some environments, especially in situations where the OpenShift cannot directly connect to the internet, require a private registry for OpenShift to pull the Cloud Pak images from. The Cloud Pak Deployer can mirror images from the entitled registry to a private registry that you want to use for the Cloud Pak(s). Also, if infrastructure which holds the OpenShift cluster is fully disconnected from the internet, the Cloud Pak Deployer can build a registry which can be stored on a portable hard disk or pen drive and then shipped to the site.
Info
Note: In all cases, the deployer can work behind a proxy to access the internet. Go to Running behind proxy for more information.
The below instructions are not limited to disconnected (air-gapped) OpenShift clusters, but are more generic for deployment using a private registry.
There are three use cases for mirroring images to a private registry and using this to install the Cloud Pak(s):
Use cases 1 and 3 are also outlined in the Cloud Pak for Data installation documentation: https://www.ibm.com/docs/en/cloud-paks/cp-data/4.5.x?topic=tasks-mirroring-images-your-private-container-registry
For specifying a private registry in the Cloud Pak Deployer configuration, please see Private registry. Example of specifying a private registry with a self-signed certificate in the configuration:
image_registry:
+- name: cpd453
+ registry_host_name: registry.coc.ibm.com
+ registry_port: 5000
+ registry_insecure: True
+The cp4d instance must reference the image_registry object using the image_registry_name:
cp4d:
+- project: zen-45
+ openshift_cluster_name: {{ env_id }}
+ cp4d_version: 4.5.3
+ openshift_storage_name: ocs-storage
+ image_registry_name: cpd453
+Info
The deployer only supports using a private registry for the Cloud Pak images, not for OpenShift itself. Air-gapped installation of OpenShift is currently not in scope for the deployer.
Warning
The registry_host_name you specify in the image_registry definition must also be available for DNS lookup within OpenShift. If the registry runs on a server that is not registered in the DNS, use its IP address instead of a host name.
The main 3 directories that are needed for both types of air-gapped installations are:
cloud-pak-deployerFpr use cases 2 and 3, where the directories must be shipped to the air-gapped cluster, the Cloud Pak Deployer and Configuration directories will be stored in the Status directory for simplicity.
This is effectively "not-air-gapped" scenario, where the following conditions apply:
The bastion server is connected to the internet and OpenShift cluster.
http_proxy, https_proxy, no_proxy environment variables)./cp-deploy.sh buildimage_registry entry to identify the private registryCreate a vault secret image-registry-<name> holding the connection credentials for the private registry specified in the configuration (image_registry). For example for a registry definition with name cpd453, create secret image-registry-cpd453.
./cp-deploy.sh vault set \
+ -vs image-registry-cpd453 \
+ -vsv "admin:very_s3cret"
+Set the environment variable for the oc login command. For example:
export CPD_OC_LOGIN="oc login api.pluto-01.coc.ibm.com:6443 -u kubeadmin -p BmxQ5-KjBFx-FgztG-gpTF3 --insecure-skip-tls-verify"
+Run the ./cp-deploy.sh env apply command to start deployment of the Cloud Pak to the OpenShift cluster. For example:
./cp-deploy.sh env apply
+image_registry definition and its reference in the cp4d definition instruct the deployer to mirror images to the private registry and to configure the OpenShift cluster to pull images from the private registry. If you have already mirrored the Cloud Pak images, you can add the --skip-mirror-images parameter to speed up the deployment process. This use case is also sometimes referred to as "semi-air-gapped", where the following conditions apply:
Warning
Please note that in this case the Cloud Pak Deployer expects an OpenShift cluster to be available already and will only work with an existing-ocp configuration. The bastion server does not have access to the internet and can therefore not instantiate an OpenShift cluster.
http_proxy, https_proxy, no_proxy environment variables)./cp-deploy.sh buildimage_registry entry to identify the private registryimage-registry-<name> holding the connection credentials for the private registry specified in the configuration (image_registry). For example for a registry definition with name cpd453, create secret image-registry-cpd453. ./cp-deploy.sh vault set \
+ -vs image-registry-cpd453 \
+ -vsv "admin:very_s3cret"
+Run the deployer using the ./cp-deploy.sh env download --skip-portable-registry command. For example:
./cp-deploy.sh env download \
+ --skip-portable-registry
+env download again. Before saving the status directory, you can optionally remove the entitlement key from the vault:
./cp-deploy.sh vault delete \
+ -vs ibm_cp_entitlement_key
+When the download finished successfully, the status directory holds the deployer scripts, the configuration directory and the deployer container image.
Ship the status directory from the internet-connected server to the bastion server.
You can use tar with gzip mode or any other compression technique. The total size of the directories should be relatively small, typically < 5 GB
The bastion server is not connected to the internet but is connected to the private registry and the OpenShift cluster.
We're using the instructions in Run on existing OpenShift, adding the --air-gapped and --skip-mirror-images flags, to start the deployer:
Untar the cloud-pak-deployer scripts, for example:
tar xvzf $STATUS_DIR/cloud-pak-deployer.tar.gz
+Set the CPD_AIRGAP environment variable to true
export CPD_AIRGAP=true
+Set the environment variable for the oc login command. For example:
export CPD_OC_LOGIN="oc login api.pluto-01.coc.ibm.com:6443 -u kubeadmin -p BmxQ5-KjBFx-FgztG-gpTF3 --insecure-skip-tls-verify"
+Run the cp-deploy.sh env apply --skip-mirror-images command to start deployment of the Cloud Pak to the OpenShift cluster. For example:
cd cloud-pak-deployer
+./cp-deploy.sh env apply \
+ --skip-mirror-images
+The CPD_AIRGGAP environment variable tells the deployer it will not download anything from the internet; --skip-mirror-images indicates that images are already available in the private registry that is included in the configuration (image_registry)
This use case is also usually referred to as "air-gapped", where the following conditions apply:
Warning
Please note that in this case the Cloud Pak Deployer expects an OpenShift cluster to be available already and will only work with an existing-ocp configuration. The bastion server does not have access to the internet and can therefore not instantiate an OpenShift cluster.
http_proxy, https_proxy, no_proxy environment variables)cp-deploy.sh buildRun the deployer using the ./cp-deploy.sh env download command. For example:
./cp-deploy.sh env download
+env download again. Before saving the status directory, you can optionally remove the entitlement key from the vault:
./cp-deploy.sh vault delete \
+ -vs ibm_cp_entitlement_key
+See the download of watsonx.ai in action: https://ibm.box.com/v/cpd-air-gapped-download
When the download finished successfully, the status directory holds the deployer scripts, the configuration directory, the deployer container image and the portable registry.
Ship the status directory from the internet-connected server to the bastion server.
You can use tar with gzip mode or any other compression technique. The status directory now holds all assets required for the air-gapped installation and its size can be substantial (100+ GB). You may want to use multi-volume tar files if you are using network transfer.
The bastion server is not connected to the internet but is connected to the private registry and OpenShift cluster.
See the air-gapped installation of Cloud Pak for Data in action: https://ibm.box.com/v/cpd-air-gapped-install. For the demonstration video, the download of the previous step has first been re-run to only download the Cloud Pak for Data control plane to avoid having to ship and upload ~700 GB.
We're using the instructions in Run on existing OpenShift, adding the CPD_AIRGAP environment variable.
Untar the cloud-pak-deployer scripts, for example:
tar xvzf $STATUS_DIR/cloud-pak-deployer.tar.gz
+cd cloud-pak-deployer
+Set the CPD_AIRGAP environment variable to true
export CPD_AIRGAP=true
+Set the environment variable for the oc login command. For example:
export CPD_OC_LOGIN="oc login api.pluto-01.coc.ibm.com:6443 -u kubeadmin -p BmxQ5-KjBFx-FgztG-gpTF3 --insecure-skip-tls-verify"
+Create a vault secret image-registry-<name> holding the connection credentials for the private registry specified in the configuration (image_registry). For example for a registry definition with name cpd453, create secret image-registry-cpd453.
./cp-deploy.sh vault set \
+ -vs image-registry-cpd453 \
+ -vsv "admin:very_s3cret"
+Run the ./cp-deploy.sh env apply command to start deployment of the Cloud Pak to the OpenShift cluster. For example:
./cp-deploy.sh env apply
+CPD_AIRGGAP environment variable tells the deployer it will not download anything from the internet. As a first action, the deployer mirrors images from the portable registry to the private registry included in the configuration (image_registry) If the Cloud Pak Deployer is run from a server that has the HTTP proxy environment variables set up, i.e. "proxy" environment variables are configured on the server and in the terminal session, it will also apply these settings in the deployer container.
The following environment variables are automatically applied to the deployer container if set up in the session running the cp-deploy.sh command:
http_proxyhttps_proxyno_proxyIf you do not want the deployer to use the proxy environment variables, you must remove them before running the cp-deploy.sh command:
unset http_proxy
+unset https_proxy
+unset no_proxy
+Specifically when running the deployer on IBM Cloud ROKS, certain OpenShift settings must be applied using DaemonSets in the kube-system namespace. Additionally, the deployer uses the oc debug node commands to retrieve kubelet and crio configuration files from the compute nodes.
The default container images used by the DaemonSets and oc debug node commands are based on Red Hat's Universal Base Image and will be pulled from Red Hat registries. This is typically not possible in air-gapped installations, hence different images must be used. It is your responsibility to copy suitable (preferably UBI) images to an image registry that is connected to the OpenShift cluster. Also, if a pull secret is needed to pull the image(s) from the registry, you must create the associated secret in the kube-system OpenShift project.
To configure alternative container images for the deployer to use, set the following properties in the .inv file kept in your configuration's inventory directory, or specify them as additional command line parameters for the cp-deploy.sh command.
If you do not set these values, the deployer assumes that the default images are used for DaemonSet and oc debug node.
| Property | Description | Example |
|---|---|---|
| cpd_oc_debug_image | Container image to be used for the oc debug command. | registry.redhat.io/rhel8/support-tools:latest |
| cpd_ds_image | Container image to be used for the DaemonSets that configure Kubelet, etc. | registry.access.redhat.com/ubi8/ubi:latest |
export CONFIG_DIR=$HOME/cpd-config && mkdir -p $CONFIG_DIR/config
+
+cat << EOF > $CONFIG_DIR/config/cpd-config.yaml
+---
+global_config:
+ environment_name: demo
+ cloud_platform: existing-ocp
+ confirm_destroy: False
+
+openshift:
+- name: cpd-demo
+ ocp_version: "4.10"
+ cluster_name: cpd-demo
+ domain_name: example.com
+ openshift_storage:
+ - storage_name: nfs-storage
+ storage_type: nfs
+
+cp4d:
+- project: cpd-instance
+ openshift_cluster_name: cpd-demo
+ cp4d_version: 4.8.3
+ accept_licenses: True
+ cartridges:
+ - name: cp-foundation
+ license_service:
+ state: disabled
+ threads_per_core: 2
+ - name: lite
+
+#
+# All tested cartridges. To install, change the "state" property to "installed". To uninstall, change the state
+# to "removed" or comment out the entire cartridge. Make sure that the "-" and properties are aligned with the lite
+# cartridge; the "-" is at position 3 and the property starts at position 5.
+#
+
+ - name: analyticsengine
+ size: small
+ state: removed
+
+ - name: bigsql
+ state: removed
+
+ - name: ca
+ size: small
+ instances:
+ - name: ca-instance
+ metastore_ref: ca-metastore
+ state: removed
+
+ - name: cde
+ state: removed
+
+ - name: datagate
+ state: removed
+
+ - name: datastage-ent-plus
+ state: removed
+
+ # The default instance is created automatically with the DataStage installation. If you want to create additional instances
+ # uncomment the section below and specify the various scaling options.
+
+ # instances:
+ # - name: ds-instance
+ # # Optional settings
+ # description: "datastage ds-instance"
+ # size: medium
+ # storage_class: efs-nfs-client
+ # storage_size_gb: 60
+ # # Custom Scale options
+ # scale_px_runtime:
+ # replicas: 2
+ # cpu_request: 500m
+ # cpu_limit: 2
+ # memory_request: 2Gi
+ # memory_limit: 4Gi
+ # scale_px_compute:
+ # replicas: 2
+ # cpu_request: 1
+ # cpu_limit: 3
+ # memory_request: 4Gi
+ # memory_limit: 12Gi
+
+ - name: db2
+ size: small
+ instances:
+ - name: ca-metastore
+ metadata_size_gb: 20
+ data_size_gb: 20
+ backup_size_gb: 20
+ transactionlog_size_gb: 20
+ state: removed
+
+ - name: db2wh
+ state: removed
+
+ - name: dmc
+ state: removed
+
+ - name: dods
+ size: small
+ state: removed
+
+ - name: dp
+ size: small
+ state: removed
+
+ - name: dv
+ size: small
+ instances:
+ - name: data-virtualization
+ state: removed
+
+ - name: hadoop
+ size: small
+ state: removed
+
+ - name: mdm
+ size: small
+ wkc_enabled: true
+ state: removed
+
+ - name: openpages
+ state: removed
+
+ - name: planning-analytics
+ state: removed
+
+ - name: rstudio
+ size: small
+ state: removed
+
+ - name: spss
+ state: removed
+
+ - name: voice-gateway
+ replicas: 1
+ state: removed
+
+ - name: watson-assistant
+ size: small
+ state: removed
+
+ - name: watson-discovery
+ state: removed
+
+ - name: watson-ks
+ size: small
+ state: removed
+
+ - name: watson-openscale
+ size: small
+ state: removed
+
+ - name: watson-speech
+ stt_size: xsmall
+ tts_size: xsmall
+ state: removed
+
+ - name: wkc
+ size: small
+ state: removed
+
+ - name: wml
+ size: small
+ state: installed
+
+ - name: wml-accelerator
+ replicas: 1
+ size: small
+ state: removed
+
+ - name: wsl
+ state: installed
+
+EOF
+Log is as a cluster administrator to be able to run the deployer with the correct permissions.
oc new-project cloud-pak-deployer
+
+oc project cloud-pak-deployer
+oc create serviceaccount cloud-pak-deployer-sa
+oc adm policy add-scc-to-user privileged -z cloud-pak-deployer-sa
+oc adm policy add-cluster-role-to-user cluster-admin -z cloud-pak-deployer-sa
+Building the deployer image typically takes ~5 minutes. Only do this if the image has not been built yet.
cat << EOF | oc apply -f -
+apiVersion: image.openshift.io/v1
+kind: ImageStream
+metadata:
+ name: cloud-pak-deployer
+spec:
+ lookupPolicy:
+ local: true
+EOF
+
+cat << EOF | oc create -f -
+kind: Build
+apiVersion: build.openshift.io/v1
+metadata:
+ generateName: cloud-pak-deployer-bc-
+ namespace: cloud-pak-deployer
+spec:
+ serviceAccount: builder
+ source:
+ type: Git
+ git:
+ uri: 'https://github.com/IBM/cloud-pak-deployer'
+ ref: wizard
+ strategy:
+ type: Docker
+ dockerStrategy:
+ buildArgs:
+ - name: CPD_OLM_UTILS_V2_IMAGE
+ value: icr.io/cpopen/cpd/olm-utils-v2:latest
+ - name: CPD_OLM_UTILS_V3_IMAGE
+ value: icr.io/cpopen/cpd/olm-utils-v3:latest
+ output:
+ to:
+ kind: ImageStreamTag
+ name: 'cloud-pak-deployer:latest'
+ triggeredBy:
+ - message: Manually triggered
+EOF
+Now, wait until the deployer image has been built.
oc get build -n cloud-pak-deployer -w
+oc create cm -n cloud-pak-deployer cloud-pak-deployer-config
+oc set data -n cloud-pak-deployer cm/cloud-pak-deployer-config \
+ --from-file=$CONFIG_DIR/config
+export CP_ENTITLEMENT_KEY=your_entitlement_key
+
+cat << EOF | oc apply -f -
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+ name: cloud-pak-deployer-status
+ namespace: cloud-pak-deployer
+spec:
+ accessModes:
+ - ReadWriteMany
+ resources:
+ requests:
+ storage: 10Gi
+EOF
+
+cat << EOF | oc apply -f -
+apiVersion: batch/v1
+kind: Job
+metadata:
+ labels:
+ app: cloud-pak-deployer
+ name: cloud-pak-deployer
+ namespace: cloud-pak-deployer
+spec:
+ parallelism: 1
+ completions: 1
+ backoffLimit: 0
+ template:
+ metadata:
+ name: cloud-pak-deployer
+ labels:
+ app: cloud-pak-deployer
+ spec:
+ containers:
+ - name: cloud-pak-deployer
+ image: cloud-pak-deployer:latest
+ imagePullPolicy: Always
+ terminationMessagePath: /dev/termination-log
+ terminationMessagePolicy: File
+ env:
+ - name: CONFIG_DIR
+ value: /Data/cpd-config
+ - name: STATUS_DIR
+ value: /Data/cpd-status
+ - name: CP_ENTITLEMENT_KEY
+ value: ${CP_ENTITLEMENT_KEY}
+ volumeMounts:
+ - name: config-volume
+ mountPath: /Data/cpd-config/config
+ - name: status-volume
+ mountPath: /Data/cpd-status
+ command: ["/bin/sh","-xc"]
+ args:
+ - /cloud-pak-deployer/cp-deploy.sh env apply -v
+ restartPolicy: Never
+ securityContext:
+ runAsUser: 0
+ serviceAccountName: cloud-pak-deployer-sa
+ volumes:
+ - name: config-volume
+ configMap:
+ name: cloud-pak-deployer-config
+ - name: status-volume
+ persistentVolumeClaim:
+ claimName: cloud-pak-deployer-status
+EOF
+The debug job can be useful if you want to access the status directory of the deployer if the deployer job has failed.
cat << EOF | oc apply -f -
+apiVersion: batch/v1
+kind: Job
+metadata:
+ labels:
+ app: cloud-pak-deployer-debug
+ name: cloud-pak-deployer-debug
+ namespace: cloud-pak-deployer
+spec:
+ parallelism: 1
+ completions: 1
+ backoffLimit: 0
+ template:
+ metadata:
+ name: cloud-pak-deployer-debug
+ labels:
+ app: cloud-pak-deployer-debug
+ spec:
+ containers:
+ - name: cloud-pak-deployer-debug
+ image: cloud-pak-deployer:latest
+ imagePullPolicy: Always
+ terminationMessagePath: /dev/termination-log
+ terminationMessagePolicy: File
+ env:
+ - name: CONFIG_DIR
+ value: /Data/cpd-config
+ - name: STATUS_DIR
+ value: /Data/cpd-status
+ volumeMounts:
+ - name: config-volume
+ mountPath: /Data/cpd-config/config
+ - name: status-volume
+ mountPath: /Data/cpd-status
+ command: ["/bin/sh","-xc"]
+ args:
+ - sleep infinity
+ restartPolicy: Never
+ securityContext:
+ runAsUser: 0
+ serviceAccountName: cloud-pak-deployer-sa
+ volumes:
+ - name: config-volume
+ configMap:
+ name: cloud-pak-deployer-config
+ - name: status-volume
+ persistentVolumeClaim:
+ claimName: cloud-pak-deployer-status
+EOF
+oc logs -f -n cloud-pak-deployer job/cloud-pak-deployer
+In some cases, especially if the OpenShift cluster is remote from where the oc command is running, the oc logs -f command may terminate abruptly.
Log is as a cluster administrator to be able to run the deployer with the correct permissions.
---
+apiVersion: v1
+kind: Namespace
+metadata:
+ creationTimestamp: null
+ name: cloud-pak-deployer
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+ name: cloud-pak-deployer-sa
+ namespace: cloud-pak-deployer
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+ name: system:openshift:scc:privileged
+ namespace: cloud-pak-deployer
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: ClusterRole
+ name: system:openshift:scc:privileged
+subjects:
+- kind: ServiceAccount
+ name: cloud-pak-deployer-sa
+ namespace: cloud-pak-deployer
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+ name: cloud-pak-deployer-cluster-admin
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: ClusterRole
+ name: cluster-admin
+subjects:
+- kind: ServiceAccount
+ name: cloud-pak-deployer-sa
+ namespace: cloud-pak-deployer
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+ name: cloud-pak-deployer-config
+ namespace: cloud-pak-deployer
+spec:
+ accessModes:
+ - ReadWriteMany
+ resources:
+ requests:
+ storage: 1Gi
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+ name: cloud-pak-deployer-status
+ namespace: cloud-pak-deployer
+spec:
+ accessModes:
+ - ReadWriteMany
+ resources:
+ requests:
+ storage: 10Gi
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: cloud-pak-deployer-wizard
+ namespace: cloud-pak-deployer
+spec:
+ replicas: 1
+ selector:
+ matchLabels:
+ app: cloud-pak-deployer-wizard
+ template:
+ metadata:
+ name: cloud-pak-deployer-wizard
+ labels:
+ app: cloud-pak-deployer-wizard
+ spec:
+ containers:
+ - name: cloud-pak-deployer
+ image: quay.io/cloud-pak-deployer/cloud-pak-deployer:latest
+ imagePullPolicy: Always
+ terminationMessagePath: /dev/termination-log
+ terminationMessagePolicy: File
+ ports:
+ - containerPort: 8080
+ protocol: TCP
+ env:
+ - name: CONFIG_DIR
+ value: /Data/cpd-config
+ - name: STATUS_DIR
+ value: /Data/cpd-status
+ - name: CPD_WIZARD_PAGE_TITLE
+ value: "Cloud Pak Deployer"
+# - name: CPD_WIZARD_MODE
+# value: existing-ocp
+ volumeMounts:
+ - name: config-volume
+ mountPath: /Data/cpd-config
+ - name: status-volume
+ mountPath: /Data/cpd-status
+ command: ["/bin/sh","-xc"]
+ args:
+ - mkdir -p /Data/cpd-config/config && /cloud-pak-deployer/cp-deploy.sh env wizard -v
+ securityContext:
+ runAsUser: 0
+ serviceAccountName: cloud-pak-deployer-sa
+ volumes:
+ - name: config-volume
+ persistentVolumeClaim:
+ claimName: cloud-pak-deployer-config
+ - name: status-volume
+ persistentVolumeClaim:
+ claimName: cloud-pak-deployer-status
+---
+apiVersion: v1
+kind: Service
+metadata:
+ name: cloud-pak-deployer-wizard-svc
+ namespace: cloud-pak-deployer
+spec:
+ selector:
+ app: cloud-pak-deployer-wizard
+ ports:
+ - nodePort: 0
+ port: 8080
+ protocol: TCP
+---
+apiVersion: route.openshift.io/v1
+kind: Route
+metadata:
+ name: cloud-pak-deployer-wizard
+spec:
+ tls:
+ termination: edge
+ to:
+ kind: Service
+ name: cloud-pak-deployer-wizard-svc
+ weight: null
+Now you can access the deployer wizard using the route created in the cloud-pak-deployer project. * Open the OpenShift console * Go to Networking → Routes * Click the Cloud Pak Deployer wizard route
Setting up a virtual machine or server to develop the Cloud Pak Deployer code. Focuses on initial setup of a server to run the deployer container, setting up Visual Studio Code, issuing GPG keys and running the deployer in development mode.
We recommend to use a Red Hat Linux server for development of the Cloud Pak Deployer, either using a virtual server in the cloud or a virtual machine on your workstation. Ideally you run Visual Studio Code on your workstation and connect it to the remote Red Hat Linux server, updating the code and running it immediately from that server.
To allow for remote development, a number of packages need to be installed on the Linux server. Not having these will cause VSCode not to work and the error messages are difficult to debug. To install these packages, run the following as the root user:
yum install -y git podman wget unzip tar gpg pinentry
+Additionally, you can also install EPEL and screen to make it easier to keep your session if it gets disconnected.
yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
+yum install -y screen
+It is recommended to use a special development user (your user name) on the Linux server, rather than using root. Not only will this be more secure; it also prevent destructive mistakes. In the below steps, we create a user fk-dev and give it sudo permissions.
useradd -G wheel fk-dev
+To give the fk-dev permissions to run commands as root, change the sudo settings.
visudo
+Scroll down until you see the following line:
# %wheel ALL=(ALL) NOPASSWD: ALL
+Change the line to look like this:
%wheel ALL=(ALL) NOPASSWD: ALL
+Now, save the file by pressing Esc, followed by : and x.
Especially when running the virtual server in the cloud, users would logon using their SSH key. This requires the public key of the workstation to be added to the development user's SSH configuration.
Make sure you run the following commands as the development user (fk-dev):
mkdir -p ~/.ssh
+chmod 700 ~/.ssh
+touch ~/.ssh/authorized_keys
+chmod 600 ~/.ssh/authorized_keys
+Then, add the public key of your workstation to the authorized_keys file.
vi ~/.ssh/authorized_keys
+Press the i to enter insert mode for vi. Then paste the public SSH key, for example:
ssh-rsa AAAAB3NzaC1yc2EAAAADAXABAAABAQEGUeXJr0ZHy1SPGOntmr/7ixmK3KV8N3q/+0eSfKVTyGbhUO9lC1+oYcDvwMrizAXBJYWkIIwx4WgC77a78....fP3S5WYgqL fk-dev
+Finally save the file by pressing Esc, followed by : and x.
Run the following commands as the development user (fk-dev):
git config --global user.name "Your full name"
+git config --global user.email "your_email_address"
+git config --global credential.helper "cache --timeout=86400"
+We also want to ensure that commits are verified (trusted) by signing them with a GPG key. This requires set up on the development server and also on your Git account.
First, set up a new GPG key:
gpg --default-new-key-algo rsa4096 --gen-key
+You will be prompted to specify your user information:
Press o at the following prompt:
Change (N)ame, (E)mail, or (O)kay/(Q)uit?
+Then, you will be prompted for a passphrase. You cannot use a passphrase for your GPG key if you want to use it for automatic signing of commits. Just press Enter multiple times until the GPG key has been generated.
List the signatures of the known keys. You will use the signature to sign the commits and to retrieve the public key.
gpg --list-signatures
+Output will look something like this:
/home/fk-dev/.gnupg/pubring.kbx
+-----------------------------------
+pub rsa4096 2022-10-30 [SC] [expires: 2024-10-29]
+ BC83E8A97538EDD4E01DC05EA83C67A6D7F71756
+uid [ultimate] FK Developer <fk-dev@ibm.com>
+sig 3 A83C67A6D7F71756 2022-10-30 FK Developer <fk-dev@ibm.com>
+You will use the signature to retrieve the public key:
gpg --armor --export A83C67A6D7F71756
+The public key will look something like below:
-----BEGIN PGP PUBLIC KEY BLOCK-----
+
+mQINBGNeGNQBEAC/y2tovX5s0Z+onUpisnMMleG94nqOtajXG1N0UbHAUQyKfirt
+O8t91ek+e5PEsVkR/RLIM1M1YkiSV4irxW/uFPucXHZDVH8azfnJjf6j6cXWt/ra
+1I2vGV3dIIQ6aJIBEEXC+u+N6rWpCOF5ERVrumGFlDhL/PY8Y9NM0cNQCbOcciTV
+5a5DrqyHC3RD5Bcn5EA0/5ISTCGQyEbJe45G8L+a5yRchn4ACVEztR2B/O5iOZbM
+.
+.
+.
+4ojOJPu0n5QLA5cI3RyZFw==
+=sx91
+-----END PGP PUBLIC KEY BLOCK-----
+Now that you have the signature, you can configure Git to sign commits:
git config --global user.signingkey A83C67A6D7F71756
+Next, add your GPG key to your Git user.
Commits done on your development server will now be signed with your user name and e-mail address and will show as Verified when listing the commits.
Clone the repository using a git command. The command below is the clone of the main Cloud Pak Deployer repository. If you have forked the repository to develop features, you will have to use the URL of your own fork.
git clone https://github.com/IBM/cloud-pak-deployer.git
+Host nickname_of_your_server
+ HostName ip_address_of_your_server
+ User fk-dev
+Once you have set up this server in the SSH config file, you can connect to it and start remote development.
cloud-pak-deployer directory (this is the cloned repository)From that point forward you can use VSCode as if you were working on your laptop, make changes and use a separate terminal to test your changes.
The Cloud Pak Deployer runs as a container on the server. When you're in the process of developing new features, having to always rebuild the image is a bit of a pain, hence we've introduced a special command line parameter.
./cp-deploy.sh env apply .... --cpd-develop [--accept-all-liceneses]
+When adding the --cpd-develop parameter to the command line, the current directory is mapped as a volume to the /cloud-pak-deployer directory within the container. This means that any latest changes you've done to the Ansible playbooks or other commands will take effect immediately.
Warning
Even though it is possible to run the deployer multiple times in parallel, for different environments, please be aware that is NOT possible when you use the --cpd-develop parameter. If you run two deploy processes with this parameters, you will see errors with permissions.
When working on multiple changes concurrently, you may have to switch between branches or tags. By default, the Cloud Pak Deployer image is built with image latest, but you can override this by setting the CPD_IMAGE_TAG environment variable in your session.
export CPD_IMAGE_TAG=cp4d-460
+./cp-deploy.sh build
+When building the deployer, the image is now tagged:
podman image ls
+REPOSITORY TAG IMAGE ID CREATED SIZE
+localhost/cloud-pak-deployer cp4d-460 8b08cb2f9a2e 8 minutes ago 1.92 GB
+When running the deployer with the same environment variable set, you will see an additional message in the output.
./cp-deploy.sh env apply
+Cloud Pak Deployer image tag cp4d-460 will be used.
+...
+By default, the cp-deploy.sh command detects if podman (preferred) or docker is found on the system. In case both are present, podman is used. You can override this behaviour by setting the CPD_CONTAINER_ENGINE environment variable.
export CPD_CONTAINER_ENGINE=docker
+./cp-deploy.sh build
+Container engine docker will be used.
+Mkdocs themes encapsulate all of the configuration and implementation details of static documentation sites. This GitHub repository has been built with a dependency on the Mkdocs tool. This GiHub repository is connected to GitHub Actions; any commit to the main branch will cause a build of the GitHub pages to be triggered. The preferred method of working while developing documentation is to use the tooling from a loacal system
If you want to test the documentation pages you're developing, it is best to run Mkdocs in a container and map your local docs folder to a folder inside the container. This avoids having to install nvm and many modules on your workstation.
Do the following:
cd docs
+./dev-doc-build.sh
+This will build a Red Hat UBI image with all requirements pre-installed. It will take ~2-10 minutes to complete this step, dependent on your network bandwidth.
./dev-doc-run.sh
+This will start the container as a daemon and tail the logs. Once running, you will see the following message:
...
+INFO - Documentation built in 3.32 seconds
+INFO - [11:55:49] Watching paths for changes: 'src', 'mkdocs.yml'
+INFO - [11:55:49] Serving on http://0.0.0.0:8000/cloud-pak-deployer/...
+Now that the container has fully started, it automatically tracks all changes under the docs folder and updates the pages site automatically. You can view the site by opening a browswer for URL:
If you don't want to test your changes locally anymore, stop the docker container.
podman kill cpd-doc
+Next time you want to test your changes, re-run the ./dev-doc-run.sh, which will delete the container, delete cache and build the documentation.
If you want to remove all from your development server, do the following:
podman rm -f cpd-doc
+podman rmi -f cpd-doc:latest
+Note that after merging your updated documentation with the main branch, the pages site will be rendered by a GitHub action. Go to GitHub Actions if you want to monitor the build process.
This document contains a few formatting rules/requirements to maintain uniformity and structure across our documentation.
Code block inputs should be created by surrounding the code text with three tick marks ``` key. For example, to create the following code block:
oc get nodes
+Your markdown input would look like:
``` { .bash .copy }
+oc get nodes
+```
+Code block outputs should specify the output language. This can be done by putting the language after the opening tick marks. For example, to create the following code block:
{
+ "cloudName": "AzureCloud",
+ "homeTenantId": "fcf67057-50c9-4ad4-98f3-ffca64add9e9",
+ "id": "d604759d-4ce2-4dbc-b012-b9d7f1d0c185",
+ "isDefault": true,
+ "managedByTenants": [],
+ "name": "Microsoft Azure Enterprise",
+ "state": "Enabled",
+ "tenantId": "fcf67057-50c9-4ad4-98f3-ffca64add9e9",
+ "user": {
+ "name": "example@example.com",
+ "type": "user"
+ }
+}
+Your markdown input would look like:
```output
+{
+ "cloudName": "AzureCloud",
+ "homeTenantId": "fcf67057-50c9-4ad4-98f3-ffca64add9e9",
+ "id": "d604759d-4ce2-4dbc-b012-b9d7f1d0c185",
+ "isDefault": true,
+ "managedByTenants": [],
+ "name": "Microsoft Azure Enterprise",
+ "state": "Enabled",
+ "tenantId": "fcf67057-50c9-4ad4-98f3-ffca64add9e9",
+ "user": {
+ "name": "example@example.com",
+ "type": "user"
+ }
+}
+```
+If you want to highlight something to reader, using an information or a warning block, use the following code:
!!! warning
+ Warning: please do not shut down the cluster at this stage.
+This will show up as:
Warning
Warning: please do not shut down the cluster at this stage.
You can also info and error.