Explore Crossplane Provider and XRDs concept

[SEQUOIIA]

• Explore XRD concept
• Create DFDS crossplane provider
• Create DFDS crossplane configuration
• Use semantic versioning for the provider so we can have a dependsOn from the configuration
• Add dependsOn for the Configuration to provider-dfds and provider-aws
  be another issue but it would be nice to get an understanding/example made)
• Add extra resources into our composite database resource (i.e we need to put a securitygroup)
• Rename our composite database resource to something better than compositepostrgesinstance
• Understand what a Claim is with regard to the provider. We have not used this yet
• Understand the parameters. the definition and the overrides.
• Is there a way to prevent a modification to a value defined in XRD (i.e allow on creation postgres or mysql, but after creation prevent switch from postgres to mysql -note: engine already is not allowed, so find another example such as applying it to allocated storage)
• ProviderConfig - we need to decide on DFDS/AWS and how we do that. We are currently relying on 'default'. Is it possible to chain through the AWS provider?

[ZaradarBH]

I spoke with Dan, he suggested that we base our provider of this repo @ https://github.com/crossplane/provider-template .

[rifisdfds]

I've started looking at the DFDS Provider from the template:

Current WIP Source is here: https://github.com/dfds/crossplane-provider-dfds/tree/provider-from-template
Container is here: https://hub.docker.com/repository/docker/dfdsdk/provider-dfds
Can be installed in sandbox by adding it to the list of providers passed to the module: crossplane_providers = ["crossplane/provider-aws:v0.18.1", "crossplane/provider-azure:v0.16.1", "dfdsdk/provider-dfds:latest"]

It is currently in a non-working state with the message cannot initialize parser backend: failed to open package stream file: open package.yaml: no such file or directory and does nothing that the template doesn't do as standard

NAME                        INSTALLED   HEALTHY   PACKAGE                             AGE
crossplane-provider-aws     True        True      crossplane/provider-aws:v0.18.1     2d23h
crossplane-provider-azure   True        True      crossplane/provider-azure:v0.16.1   21h
dfdsdk-provider-dfds        True        False     dfdsdk/provider-dfds:latest         21h

I suspect because the template is out of date and not v2 as per this issue but not confirmed:
crossplane/provider-template#5

I will look at existing providers such as aws to see if I can work out how they work and get ours in a healthy state

[rifisdfds]

I have managed to get the provider in a healthy state:

NAME                        INSTALLED   HEALTHY   PACKAGE                             AGE
crossplane-provider-aws     True        True      crossplane/provider-aws:v0.18.1     6d1h
crossplane-provider-azure   True        True      crossplane/provider-azure:v0.16.1   4d
dfdsdk-provider-dfds        True        True      dfdsdk/provider-dfds:latest         102m

We now have the provider controller in this docker repo: https://hub.docker.com/repository/docker/dfdsdk/provider-dfds-controller

The package is still in: https://hub.docker.com/repository/docker/dfdsdk/provider-dfds

Some instructions to follow are:

To build and publish:

For dfdsdk\provider-dfds-controller:

make build
make image
make image-push

For dfdsdk\provider-dfds:

cd package
kubectl crossplane build provider
kubectl crossplane push provider dfdsd/provider-dfds:latest

To install in cluster:

kubectl crossplane install provider dfdsdk/provider-dfds:latest

This will need putting into an azure-pipelines.yaml and an automated build setting up on dockerhub but I am not worrying about that for now

[rifisdfds]

Next step is to get some XRD CRDs' made after some investigation into how to do those and include them in the provider. We can either create some around some simple resources like RDS (including the SecurityGroup) or use Inventa to start

[rifisdfds]

We have created and deployed an XRD today which involves a CompositeResourceDefinition(XRD), a Composition using that XRD, and a CompositeInstance which deploys that resource (aws rdsinstance). It deployed successfully, modified successfully with updated parameters and destroyed successfully so we have a rudimentary understanding we can build on. It was done using an AWS RDSInstance (postgres).

We have moved onto the task of putting this into our DFDS Provider but it seems more complicated to do this than it first appeared so we have a lot of reading and experimenting to do to make progress, given how little/no documentation there is

[rifisdfds]

We now have a DFDS Provider package, a DFDS Configuration package, and a Composite resource working. We need to do some extra tasks to make this properly usable/presentable.

• Use semantic versioning for the provider so we can have a dependsOn from the configuration
• Add extra resources into our composite database resource (i.e we need to put a securitygroup in, vpc etc)
• Rename our composite database resource to something better than compositepostrgesinstance
• ProviderConfig - we need to decide on DFDS/AWS and how we do that. We are currently relying on 'default'. Is it possible to chain through the AWS provider?
• We need an example deployment that shows the minimal requirement to deploy a resource
• We need to look into RBAC around XRD so that we can do multi-tenancy single cluster appropriately (This perhaps should be another issue but it would be nice to get an understanding/example made)
• Understand what a Claim is with regard to the provider. We have not used this yet

[samidbb]

Need to upgrade to 1.13 to enable CombineFromComposite property in composition files

[samidbb]

To enable advanced policies like preventing switching engineVersion or prevent downgrading and displaying error messages directly to claim users, OPA Gatekeeper https://www.openpolicyagent.org/docs/latest/kubernetes-introduction/ watch https://www.youtube.com/watch?v=TaF0_syejXc for inspiration