Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added OpenShift GCP supplemental values #237

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from
Draft

Added OpenShift GCP supplemental values #237

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
062274b
add gcp supplemental values
Dec 11, 2024
518e575
add gcp supplemental values
Dec 11, 2024
c9c151e
fix: split the docs into gcp and os
Dec 12, 2024
ddbe56b
add comments
Dec 13, 2024
fe4c31f
resolve comments
Dec 16, 2024
c722457
add comments
Dec 17, 2024
0b9df67
fix comments and remove dfs
Dec 17, 2024
228753e
fix comments
Dec 18, 2024
bdb62ad
change database config for grafana
Dec 18, 2024
4c0d10b
fix: grafana database
Dec 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions getting-started/templates/GCP/Openshift/openshift_supplemental_values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
## Openshift prohibits running the pods as a root user. The redis and rabbitmq pods should be run as non-root users.
priyadarshini-ni marked this conversation as resolved.
Show resolved Hide resolved
## Disabling 'adaptSecurityContext' will restrict openshift from adapting security context values for the mentioned pods.
rabbitmq:
global:
compatibility:
openshift:
priyadarshini-ni marked this conversation as resolved.
Show resolved Hide resolved
adaptSecurityContext: disabled

webserver:
redis-cluster:
global:
compatibility:
openshift:
adaptSecurityContext: disabled
376 changes: 376 additions & 0 deletions getting-started/templates/GCP/gcp-supplemental-values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,376 @@
global:
ingress:
api:
annotations:
kubernetes.io/ingress.class: <user-defined-ingress-class> #<ATTENTION> Enter the ingress class used
ui:
annotations:
kubernetes.io/ingress.class: <user-defined-ingress-class> #<ATTENTION> Enter the ingress class used

testmonitorservice:
## Configuring sidecars for the Test Monitor Service
## ref: https://cloud.google.com/sql/docs/postgres/connect-kubernetes-engine#run_the_in_a_sidecar_pattern
priyadarshini-ni marked this conversation as resolved.
Show resolved Hide resolved
sidecars:
priyadarshini-ni marked this conversation as resolved.
Show resolved Hide resolved
- name: cloud-sql-auth-proxy
image: gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.8.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets remove the tag from here and replace with a placeholder and attention comment

volumeMounts:
# This volume mount is required for the proxy to authenticate with cloudSQL using a service account key file.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this token volume mount only required for workload identity?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, added comment for the same

- name: <token-volume-name> #<ATTENTION> Enter the volume name where the token is mounted
mountPath: <token-mount-path> #<ATTENTION> Enter the path where the token is mounted
readOnly: true
# This volume mount is required for the proxy to authenticate with cloudSQL when using Workload Identity Federation.
priyadarshini-ni marked this conversation as resolved.
Show resolved Hide resolved
- name: <config-volume-name> #<ATTENTION> Enter the volume name where config.json is mounted
mountPath: /secrets/
readOnly: true
env:
- name: "GOOGLE_APPLICATION_CREDENTIALS"
value: /secrets/<secret-key> #<ATTENTION> Enter the key which was used while creating the secret
args:
# If connecting from a VPC-native GKE cluster, you can use the
# following flag to have the proxy connect over private IP
- "--private-ip"

# If you are not connecting with Automatic IAM, you can delete the following flag.
- "--auto-iam-authn"

# Enable structured logging with LogEntry format:
- "--structured-logs"

# Ensures the proxy exits gracefully with a 0 exit code when it receives a SIGTERM signal
- "--exit-zero-on-sigterm"

- "--port=5432"
- "<connection-name>" #<ATTENTION> Enter the connection name from the CloudSQL instance

# The credentials file is required for the proxy to authenticate using a service account key file.
priyadarshini-ni marked this conversation as resolved.
Show resolved Hide resolved
- "--credentials-file=/secrets/<secret-key>" #<ATTENTION> Enter the key which was used while creating the secret
securityContext:
# The default Cloud SQL Auth Proxy image runs as the
# "nonroot" user and group (uid: 65532) by default.
runAsNonRoot: true
restartPolicy: Always
# You should use resource requests/limits as a best practice to prevent
# pods from consuming too many resources and affecting the execution of
# other pods. You should adjust the following values based on what your
# application needs. For details, see
# https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
resources:
requests:
# The proxy's memory use scales linearly with the number of active
# connections. Fewer open connections will use less memory. Adjust
# this value based on your application's requirements.
memory: "1Gi"
# The proxy's CPU use scales linearly with the amount of IO between
# the database and the application. Adjust this value based on your
# application's requirements.
cpu: "1"

## Extra volumes that can be used in sidecars
extraVolumes:
# This volume is required for the proxy to authenticate with cloudSQL using a service account key file.
- name: <token-volume-name> #<ATTENTION> Enter the volume name where the token is to be mounted
projected:
sources:
- serviceAccountToken:
audience: <audience-name> #<ATTENTION> Enter the audience name for the projected service account token
expirationSeconds: 3600
path: token
# This volume is required for the proxy to authenticate with cloudSQL when using Workload Identity Federation.
- name: <config-volume-name> #<ATTENTION> Enter the volume name where config.json is to be mounted
secret:
secretName: <secret-name> #<ATTENTION> Enter the secret name where config.json going to be added.

connectionInfo:
## @param database.connectionInfo.host PostgreSQL hostname. Since the cloud-sql-auth-proxy is used, this value should be localhost.
##
host: "localhost"
## @param database.connectionInfo.port PostgreSQL port
##
port: "5432"
## @param database.connectionInfo.dbName PostgreSQL database name
##
dbName: <database-name>
## @param database.connectionInfo.user PostgreSQL username used by the service
##
user: <username>
## @param database.connectionInfo.secretName The name of an existing secret with
## PostgreSQL connection credentials
##
secretName: <secret-name>
## @param database.connectionInfo.passwordKey Password key for database.connectionInfo.user to be retrieved from existing secret
## NOTE: Ignored unless `database.connectionInfo.secretName` parameter is set.
##
# passwordKey: "passkey"
## @param database.connectionInfo.migrationPasswordKey Password key for database.connectionInfo.migrationUser to be retrieved from existing secret
## If unset database.connectionInfo.passwordKey is used instead.
## NOTE: Ignored unless `database.connectionInfo.secretName` parameter is set.
##
migrationPasswordKey: null

serviceAccount:
## @param serviceAccount.annotations Annotations to add to the service account
##
annotations:
{
iam.gke.io/gcp-service-account=YOUR-GSA-NAME@YOUR-GOOGLE-CLOUD-PROJECT.iam.gserviceaccount.com,
}
priyadarshini-ni marked this conversation as resolved.
Show resolved Hide resolved
## @param serviceAccount.name The name of the service account to use.
## If not set and create is true, a name is generated using the fullname template
priyadarshini-ni marked this conversation as resolved.
Show resolved Hide resolved
##
name: ""

dashboardhost:
grafana:
## Configuring extraContainers for the Dashboardhost service
priyadarshini-ni marked this conversation as resolved.
Show resolved Hide resolved
## ref: https://cloud.google.com/sql/docs/postgres/connect-kubernetes-engine#run_the_in_a_sidecar_pattern
extraContainers: |
- name: cloud-sql-auth-proxy
image: gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.8.0
volumeMounts:
# This volume mount is required for the proxy to authenticate with cloudSQL using a service account key file.
- name: <token-volume-name> #<ATTENTION> Enter the volume name where the token is mounted
mountPath: <token-mount-path> #<ATTENTION> Enter the path where the token is mounted
readOnly: true
# This volume mount is required for the proxy to authenticate with cloudSQL when using Workload Identity Federation.
- name: <config-volume-name> #<ATTENTION> Enter the volume name where config.json is mounted
mountPath: /secrets/
readOnly: true
env:
- name: "GOOGLE_APPLICATION_CREDENTIALS"
value: /secrets/<secret-key> #<ATTENTION> Enter the key which was used while creating the secret
args:
# If connecting from a VPC-native GKE cluster, you can use the
# following flag to have the proxy connect over private IP
- "--private-ip"

# If you are not connecting with Automatic IAM, you can delete the following flag.
- "--auto-iam-authn"

# Enable structured logging with LogEntry format:
- "--structured-logs"

# Ensures the proxy exits gracefully with a 0 exit code when it receives a SIGTERM signal
- "--exit-zero-on-sigterm"

- "--port=5432"
- "<connection-name>" #<ATTENTION> Enter the connection name from the CloudSQL instance

# The credentials file is required for the proxy to authenticate using a service account key file.
- "--credentials-file=/secrets/<secret-key>" #<ATTENTION> Enter the key which was used while creating the secret
securityContext:
# The default Cloud SQL Auth Proxy image runs as the
# "nonroot" user and group (uid: 65532) by default.
runAsNonRoot: true
restartPolicy: Always
# You should use resource requests/limits as a best practice to prevent
# pods from consuming too many resources and affecting the execution of
# other pods. You should adjust the following values based on what your
# application needs. For details, see
# https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
resources:
requests:
# The proxy's memory use scales linearly with the number of active
# connections. Fewer open connections will use less memory. Adjust
# this value based on your application's requirements.
memory: "1Gi"
# The proxy's CPU use scales linearly with the amount of IO between
# the database and the application. Adjust this value based on your
# application's requirements.
cpu: "1"

## Extra volumes that can be used in extrContainers
extraContainerVolumes:
# This volume is required for the proxy to authenticate with cloudSQL using a service account key file.
- name: <token-volume-name> #<ATTENTION> Enter the volume name where the token is to be mounted
projected:
sources:
- serviceAccountToken:
audience: <audience-name> #<ATTENTION> Enter the audience name for the projected service account token
expirationSeconds: 3600
path: token
# This volume is required for the proxy to authenticate with cloudSQL when using Workload Identity Federation.
- name: <config-volume-name> #<ATTENTION> Enter the volume name where config.json is to be mounted
secret:
secretName: <secret-name> #<ATTENTION> Enter the secret name where config.json going to be added.

connectionInfo:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like these are not the right values for grafana.

Check here

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The values for grafana database have been modified

## @param database.connectionInfo.host PostgreSQL hostname. Since the cloud-sql-auth-proxy is used, this value should be localhost.
##
host: "localhost"
## @param database.connectionInfo.port PostgreSQL port
##
port: "5432"
## @param database.connectionInfo.dbName PostgreSQL database name
##
dbName: <database-name>
priyadarshini-ni marked this conversation as resolved.
Show resolved Hide resolved
## @param database.connectionInfo.user PostgreSQL username used by the service
##
user: <username>
## @param database.connectionInfo.secretName The name of an existing secret with
## PostgreSQL connection credentials
##
secretName: <secret-name>
## @param database.connectionInfo.passwordKey Password key for database.connectionInfo.user to be retrieved from existing secret
## NOTE: Ignored unless `database.connectionInfo.secretName` parameter is set.
##
# passwordKey: "passkey"
## @param database.connectionInfo.migrationPasswordKey Password key for database.connectionInfo.migrationUser to be retrieved from existing secret
## If unset database.connectionInfo.passwordKey is used instead.
## NOTE: Ignored unless `database.connectionInfo.secretName` parameter is set.
##
migrationPasswordKey: null

serviceAccount:
## @param serviceAccount.annotations Annotations to add to the service account
##
annotations:
{
iam.gke.io/gcp-service-account=YOUR-GSA-NAME@YOUR-GOOGLE-CLOUD-PROJECT.iam.gserviceaccount.com,
}
## @param serviceAccount.name The name of the service account to use.
## If not set and create is true, a name is generated using the fullname template
##
name: ""

dynamicformfields:
## Configuring sidecars for the Dynamic Form Fields Service
## ref: https://cloud.google.com/sql/docs/postgres/connect-kubernetes-engine#run_the_in_a_sidecar_pattern
sidecars:
- name: cloud-sql-auth-proxy
image: gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.8.0
volumeMounts:
# This volume mount is required for the proxy to authenticate with cloudSQL using a service account key file.
- name: <token-volume-name> #<ATTENTION> Enter the volume name where the token is mounted
mountPath: <token-mount-path> #<ATTENTION> Enter the path where the token is mounted
readOnly: true
# This volume mount is required for the proxy to authenticate with cloudSQL when using Workload Identity Federation.
- name: <config-volume-name> #<ATTENTION> Enter the volume name where config.json is mounted
mountPath: /secrets/
readOnly: true
env:
- name: "GOOGLE_APPLICATION_CREDENTIALS"
value: /secrets/<secret-key> #<ATTENTION> Enter the key which was used while creating the secret
args:
# If connecting from a VPC-native GKE cluster, you can use the
# following flag to have the proxy connect over private IP
- "--private-ip"

# If you are not connecting with Automatic IAM, you can delete the following flag.
- "--auto-iam-authn"

# Enable structured logging with LogEntry format:
- "--structured-logs"

# Ensures the proxy exits gracefully with a 0 exit code when it receives a SIGTERM signal
- "--exit-zero-on-sigterm"

- "--port=5432"
- "<connection-name>" #<ATTENTION> Enter the connection name from the CloudSQL instance

# The credentials file is required for the proxy to authenticate using a service account key file.
- "--credentials-file=/secrets/<secret-key>" #<ATTENTION> Enter the key which was used while creating the secret
securityContext:
# The default Cloud SQL Auth Proxy image runs as the
# "nonroot" user and group (uid: 65532) by default.
runAsNonRoot: true
restartPolicy: Always
# You should use resource requests/limits as a best practice to prevent
# pods from consuming too many resources and affecting the execution of
# other pods. You should adjust the following values based on what your
# application needs. For details, see
# https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
resources:
requests:
# The proxy's memory use scales linearly with the number of active
# connections. Fewer open connections will use less memory. Adjust
# this value based on your application's requirements.
memory: "1Gi"
# The proxy's CPU use scales linearly with the amount of IO between
# the database and the application. Adjust this value based on your
# application's requirements.
cpu: "1"

## Extra volumes that can be used in sidecars
extraVolumes:
# This volume is required for the proxy to authenticate with cloudSQL using a service account key file.
- name: <token-volume-name> #<ATTENTION> Enter the volume name where the token is to be mounted
projected:
sources:
- serviceAccountToken:
audience: <audience-name> #<ATTENTION> Enter the audience name for the projected service account token
expirationSeconds: 3600
path: token
# This volume is required for the proxy to authenticate with cloudSQL when using Workload Identity Federation.
- name: <config-volume-name> #<ATTENTION> Enter the volume name where config.json is to be mounted
secret:
secretName: <secret-name> #<ATTENTION> Enter the secret name where config.json going to be added.
priyadarshini-ni marked this conversation as resolved.
Show resolved Hide resolved

connectionInfo:
## @param database.connectionInfo.host PostgreSQL hostname. Since the cloud-sql-auth-proxy is used, this value should be localhost
##
host: "localhost"
## @param database.connectionInfo.port PostgreSQL port
##
port: "5432"
## @param database.connectionInfo.dbName PostgreSQL database name
##
dbName: <database-name>
## @param database.connectionInfo.user PostgreSQL username used by the service
##
user: <username>
## @param database.connectionInfo.secretName The name of an existing secret with
## PostgreSQL connection credentials
##
secretName: <secret-name>
## @param database.connectionInfo.passwordKey Password key for database.connectionInfo.user to be retrieved from existing secret
## NOTE: Ignored unless `database.connectionInfo.secretName` parameter is set.
##
# passwordKey: "passkey"
## @param database.connectionInfo.migrationPasswordKey Password key for database.connectionInfo.migrationUser to be retrieved from existing secret
## If unset database.connectionInfo.passwordKey is used instead.
## NOTE: Ignored unless `database.connectionInfo.secretName` parameter is set.
##
migrationPasswordKey: null

serviceAccount:
## @param serviceAccount.annotations Annotations to add to the service account
##
annotations:
{
iam.gke.io/gcp-service-account=YOUR-GSA-NAME@YOUR-GOOGLE-CLOUD-PROJECT.iam.gserviceaccount.com,
}
## @param serviceAccount.name The name of the service account to use.
## If not set and create is true, a name is generated using the fullname template
##
name: ""

fileingestion:
priyadarshini-ni marked this conversation as resolved.
Show resolved Hide resolved
s3:
port: 443
bucket: <name-of-gcs-bucket> #<ATTENTION> Enter the name of the GCS bucket for fileingestion service
scheme: "https://"
host: "storage.googleapis.com"

feedservice:
s3:
port: 443
bucket: <name-of-gcs-bucket> #<ATTENTION> Enter the name of the GCS bucket for feedservice service
scheme: "https://"
host: "storage.googleapis.com"

saltmaster:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this expected to change based on the cloud provider?

serviceTCP:
type: LoadBalancer

nbexecservice:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nbexec (has direct s3 dependency), dfs and dremio in dfs too have GCS dependency. Can we also include them?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added s3 dependency for nbexecservice. The GCS guide has not been included in this PR for DFS as the support is not fully available yet. Refer PR description for more info.

argo:
## Configure S3/MinIO access.
priyadarshini-ni marked this conversation as resolved.
Show resolved Hide resolved
##
artifactRepository:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s3 dependency for Argo in nbexec is no longer required. We can remove this

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

s3:
bucket: <name-of-gcs-bucket> #<ATTENTION> Enter the name of the GCS bucket for nbexecservice service
port: 443
region: "us-east1"
priyadarshini-ni marked this conversation as resolved.
Show resolved Hide resolved
insecure: false
scheme: "https://"
host: "storage.googleapis.com"
Loading