Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support user configs, user secrets and separate environments for cassandra and sidecar #218

Merged
merged 9 commits into from
Aug 21, 2019
Merged

Support user configs, user secrets and separate environments for cassandra and sidecar #218

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
c8cdba9
Initial work for clouds secrets and user-defined configmap
alourie Jul 26, 2019
9f8b223
PR comments fixes
alourie Jul 31, 2019
e7109f6
Support TLS certificates for internal communication
alourie Aug 2, 2019
f3f49bb
PR comments, docs updates
alourie Aug 5, 2019
a38eb32
Cleanups
alourie Aug 5, 2019
1af32c3
Path naming update
alourie Aug 6, 2019
fa47f90
Cleanups
alourie Aug 20, 2019
a860bbe
Merge branch 'superdupertopsecretrewrite' of github.com:instaclustr/c…
alourie Aug 20, 2019
aff1123
go fmt
alourie Aug 20, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions deploy/crds/cassandraoperator_v1alpha1_cassandradatacenter_crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ spec:
type: object
spec:
properties:
backupSecretVolume:
type: object
cassandraImage:
type: string
cluster:
Expand All @@ -35,6 +37,10 @@ spec:
type: string
dataVolumeClaimSpec:
type: object
env:
items:
type: object
type: array
imagePullPolicy:
type: string
imagePullSecrets:
Expand All @@ -44,12 +50,18 @@ spec:
nodes:
format: int32
type: integer
privilegedSupported:
type: boolean
prometheusSupport:
type: boolean
resources:
type: object
sidecarImage:
type: string
userConfigMapVolumeSource:
type: object
userSecretVolume:
type: object
required:
- nodes
- cassandraImage
Expand Down
25 changes: 25 additions & 0 deletions examples/go/example-datacenter.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,31 @@ spec:
cassandraImage: "gcr.io/cassandra-operator/cassandra:3.11.3"
sidecarImage: "gcr.io/cassandra-operator/cassandra-sidecar:latest"
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: regcred
backupSecretVolume:
secretName: backup-secret
# type is a workaround for https://github.com/kubernetes/kubernetes/issues/68466
type: array
items:
- key: creds.json
path: creds.json
env:
- name: GOOGLE_APPLICATION_CREDENTIALS
value: "/etc/google/creds.json"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the "default"? If so, do we need to set it as an environment variable?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There does not seem to be any default https://cloud.google.com/docs/authentication/getting-started

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i checked the source code, there is not any default location

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, no defaults for that one. This is just an example of where to place that file, I literally look it up in the env and if it's there - we use it. If not defined, we'll use /etc/gcp, but I might change that to something more generic.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the default process is this: https://cloud.google.com/docs/authentication/production#finding_credentials_automatically

So yes, we need a way to define the environment variable, but that's probably also going to be the more uncommon approach, as on GKE the credentials will be automatically discovered.

C* clusters running on GKE would most likely backup to Google Cloud Storage. Same on Amazon EKS -- S3 is the most likely destination. But we don't want to prevent clusters running on other K8s environments from accessing cloud storage, so having the option is always nice. Could also be useful for GKE -> EKS migrations for example.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, it's a convenient option, and it's optional. Also, might be that even on GKE they might want to use some different account for backups, and that's when it would be handy.

userSecretVolume:
secretName: dc1-user-secret
userConfigMapVolumeSource:
name: dc1-user-config
alourie marked this conversation as resolved.
Show resolved Hide resolved
# type is a workaround for https://github.com/kubernetes/kubernetes/issues/68466
type: array
items:
- key: cassandra_ssl
path: cassandra.yaml.d/003-ssl.yaml
- key: cqlshrc
path: cqlshrc
- key: install_cqlshrc
path: cassandra-env.sh.d/003-install-cqlshrc.sh
alourie marked this conversation as resolved.
Show resolved Hide resolved
resources:
limits:
memory: 1Gi
Expand Down
26 changes: 14 additions & 12 deletions pkg/apis/cassandraoperator/v1alpha1/cassandradatacenter.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,20 @@ import (
type CassandraDataCenterSpec struct {
// Cluster is either a string or v1.LocalObjectReference
//Cluster interface{} `json:"cluster,omitempty"`
Cluster string `json:"cluster,omitempty"`
Nodes int32 `json:"nodes"`
CassandraImage string `json:"cassandraImage"`
SidecarImage string `json:"sidecarImage"`
ImagePullPolicy v1.PullPolicy `json:"imagePullPolicy"`
ImagePullSecrets []v1.LocalObjectReference `json:"imagePullSecrets,omitempty"`

Resources v1.ResourceRequirements `json:"resources"`

DataVolumeClaimSpec v1.PersistentVolumeClaimSpec `json:"dataVolumeClaimSpec"`

PrometheusSupport bool `json:"prometheusSupport"`
Cluster string `json:"cluster,omitempty"`
Nodes int32 `json:"nodes"`
CassandraImage string `json:"cassandraImage"`
SidecarImage string `json:"sidecarImage"`
ImagePullPolicy v1.PullPolicy `json:"imagePullPolicy"`
ImagePullSecrets []v1.LocalObjectReference `json:"imagePullSecrets,omitempty"`
BackupSecretVolume v1.SecretVolumeSource `json:"backupSecretVolume,omitempty"`
alourie marked this conversation as resolved.
Show resolved Hide resolved
UserSecretVolume v1.SecretVolumeSource `json:"userSecretVolume,omitempty"`
UserConfigMapVolumeSource v1.ConfigMapVolumeSource `json:"userConfigMapVolumeSource,omitempty"`
Resources v1.ResourceRequirements `json:"resources"`
DataVolumeClaimSpec v1.PersistentVolumeClaimSpec `json:"dataVolumeClaimSpec"`
PrivilegedSupported bool `json:"privilegedSupported,omitempty"`
PrometheusSupport bool `json:"prometheusSupport"`
Env []v1.EnvVar `json:"env,omitempty"`
}

// CassandraDataCenterStatus defines the observed state of CassandraDataCenter
Expand Down
33 changes: 31 additions & 2 deletions pkg/apis/cassandraoperator/v1alpha1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

35 changes: 34 additions & 1 deletion pkg/apis/cassandraoperator/v1alpha1/zz_generated.openapi.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions pkg/controller/cassandradatacenter/configmap.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ func createOrUpdateOperatorConfigMap(rctx *reconciliationRequestContext, seedNod

addPrometheusSupport(rctx.cdc, addFileFn)


if err := controllerutil.SetControllerReference(rctx.cdc, configMap, rctx.scheme); err != nil {
return err
}
Expand Down
Loading