NuoDB Control Plane 2.7.0

Changelog

New features

• Added automatic TLS key rotation for Domain and Database custom resources.
• Added /events REST endpoints for DBaaS resources, which stream resource updates as server-sent events.
• Added --watch option for nuodb-cp <resource> get subcommands, which streams updates to the resource.
• Added DBaaS External Operator to allow declarative management of DBaaS resources via the REST API.
• Added retention.settings to the backup policy resource exposed by the REST API, allowing various aspects of the retention policy to be controlled.
• Added status.retainedAs field to the backup resource by the REST API, listing the retention cycles that apply to the backup.
• Added the ability to propagate certain labels to resources verbatim via the REST API.
• Changed the default repository for NuoDB images from docker.io/nuodb/nuodb-ce to docker.io/nuodb/nuodb.

Resolved issues

• Fixed issues related to backups and backup policies.

Commits

• Add properties to control reconciliation
• Database TLS certificates rotation may complete prematurely
• Omit certain labels when propagating to Helm resources
• Minor backup fixes caused by client cache misses
• Add config options for External Operator
• Record backups not scheduled by a backup policy
• Fix failed backups retention
• Inject size constraints into OpenAPI schemas
• Fix empty ConfigMap key
• Perform retry on unpublish DBA password
• Generate and publish DBA password on database creation
• Allow publishing of resource fields to Secrets, ConfigMaps
• Finalize backup even if backup handle is missing
• Relax database backup validation inside webhook
• Create normal event for backup when database is disabled
• Add passthroughLabelKeyPrefixes to claim CRDs
• Enable Hotsnap in backup hooks via Helm feature
• Add validation constraints and additional printer columns to CRDs
• Add nuodb-cp subcommands, Helm chart for External Operator
• Replace nuodb/nuodb-ce with nuodb/nuodb
• Make id field in event message relative to request path
• Certificate rotation for NuoDB database
• Add DBaaS external operator (claim operator)
• Reconcile target on Domain update
• Add watchAll query parameter to events endpoints
• make openapi URL available to all authenticated users
• Refactor events framework to remove code duplication
• Certificate rotation for NuoDB domain
• Added retainedAs field to backup status
• Fix policy retention for multiple databases
• Add --watch to nuodb-cp <resource> get subcommands
• Add event publishing for all resources
• Allow labels with special prefixes to be passed through
• Revert picocli to avoid bug displaying duplicate help text
• Support events for lists of resources
• Expose backup rotation settings in REST API
• Add event streaming framework

NuoDB Control Plane 2.6.1

Changelog

• Revert picocli to avoid bug displaying duplicate help text
• Fix E2E test using old NuoDB image tag
• Fix policy retention for multiple databases
• Bump version in OpenAPI spec
• Update version to 2.6.1

NuoDB Control Plane 2.6.0

Changelog

New features

• Added DatabaseBackupPolicy CRD and controller that enables backup scheduling and retention policies, which retains backups at descending granularity based on age.
• Added /backuppolicies REST resource that exposes CRUD operations on backup policies (DatabaseBackupPolicy).
• Added nuodb-cp backuppolicy subcommands that allow backup policies to be inspected and managed using the nuodb-cp command-line tool.
• Added the status.createdByPolicy field to backup resources exposed by the REST API, which contains the name of the backup policy that created it.
• Inject Kubernetes labels cp.nuodb.com/sla and cp.nuodb.com/tier that allow resources to be filtered on SLA and tier.
• Added the ability to use Jetty as an alternative HTTP server implementation.
• Allow extra Kubernetes labels to be injected for workloads and pods generated by the DBaaS Control Plane.
• Added retry of PATCH requests that fail due to a concurrent updates, which would previously generate 409 Conflict / CONCURRENT_UPDATE errors that would be retried by the client.
• Ensure that the Ready=true status condition is always based on the latest configuration (spec) of Domain and Database resources.
• On helm install of the nuodb-cp-operator Helm chart, disable snapshot-based backups if the VolumeSnapshot and VolumeSnapshotContent CRDs are not available.

Resolved issues

• DBaaS user authentication tokens were exposed in the request log for the REST service, which would have been accessible to anyone with kubectl logs access to the REST service pods.
• Ingress traffic for the HTTP server exposed by the backup-hooks sidecar container was disabled in environments that used NetworkPolicy to control traffic flow.
• 409 Conflict errors from the HelmApp resource associated with Domain or Database were not retried.
• Database quota statistics were incorrect if an update was performed on the matching criteria.

Commits

• Support SD-HELM service definition
• Backup policy retention in controller
• Add resourceLabels and podLabels Helm values
• Backup rotation scheme and retention validation
• Add nuodb-cp backuppolicy subcommands
• Add Token Authentication doc
• Add status.createdByPolicy to backup resource
• Add retry to PATCH /backuppolicy and other improvements
• Add /backuppolicy sub-resources for /databases and /backups
• Transition Database to unready during rolling upgrade
• Perform retry on PATCH requests
• Add /backuppolicies resource
• Inject labels for SLA and tier
• Disable backup manager if VolumeSnapshot CRD not available
• Add ingress rule for backup-hooks to network policy
• Do not expose token in request log
• Fix isConflict to detect wrapped errors
• Adjust resource usage on update only if the old resource match the quota
• Add service provider interface for server implementation
• Database backup policy scheduling

NuoDB Control Plane 2.5.0

Changelog

New features

• The REST API GET endpoints used to list resources can now return the payload fields of the resources by specifying the expand query parameter, and paging has been added to the list APIs which is controlled by the limit and offset/cursor query parameters.
• The initial invocation of nuodb-cp made within a Docker container is now faster due to pre-populated JIT caches in the image build.
• The REST API GET /openapi endpoint has been added that allows the user to obtain the OpenAPI specification of the REST API.
• To allow newer client versions to detect that the request issued is not supported by the REST service, the UNKNOWN_REQUEST API error code has been added.
• The POST /databases/{org}/{proj}/{db}/dbaPassword endpoint has been added to allow the user to update the DBA password for a database.
• The --restore-from-backup argument has been added to nuodb-cp database create to allow users to restore/clone a database from a backup.
• DBaaS operators/administrators can now pin the Helm charts version based on SLA.
• The ability to create authentication tokens with constrained access and expiration is now generally available.
  • On helm install of the nuodb-cp-rest Helm chart, the secret key used to create authentication tokens by the REST service is automatically generated unless one already exists.
  • The user can specify an authentication token when invoking nuodb-cp subcommands by using the --token argument or the NUODB_CP_TOKEN environment variable.
• For all DBaaS resources exposed by REST API, the PUT method now returns the resource content immediately after creation or update.

Resolved issues

• Backups with fully-qualified names longer than 63 characters could not be restored.
• Failed attempts to restore a backup were retried indefinitely by the DBaaS controller with a small fixed delay, generating concurrent update errors when the user attempted to manually update the database configuration.
• Some validation errors could generate unexpected response codes based on whether the error was encountered in the mutating webhook.
• Database quotas could have inconsistent statistics if a database is rapidly created and deleted.
• Concurrent backup requests could result in failures that would leave one of the backups permanently in a Failed state.
• A backup requested while the DBaaS controller was not running would not be reconciled when the controller restarted.
• Backup resources could enter a Failed state due to transient concurrent update errors that occur internally.
• A project or database could be reported as Modifying immediately after being created.
• A user access check is performed on the backup resource referenced in the restoreFrom field of a database before creation. This access check could fail even if the user does have access to the backup, if the backup was specified using a relative resource path.
• Newer versions of nuodb-cp could create requests with fields unknown to older versions of the REST service, even if they were not specified by the user. This has been resolved by omitting fields that have null values.
• It was not possible to use nuodb-cp user update to grant a user access to additional organizations. The --allow-cross-organization flag has been added to nuodb-cp user update so that additional organizations can be added to the accessRule.allow list when editing the resource.

Other changes

• The undocumented status.conditions field of project and database resources has been removed.

Commits

• Allow JWT token to be used in nuodb-cp
• Remove undocumented status conditions from REST models
• Add payload expansion and paging to list APIs
• Allow backups with large names to be restored
• Allow limiting database restore attempts
• Defer some defaulting errors until validation
• Revert immutable storageClass Helm value for archive and journal volumes
• Expire all quota enforcement records
• Optimize REST server start-up and update base image
• Expose spec for running server and add UNKNOWN_REQUEST code
• Add REST API to update DBA password
• Retry backup hooks on failure
• Trigger backup reconciliation on operator restart
• Fixing various backup issues
• Add --restore-from-backup to nuodb-cp database create
• Fix panic when propagating Disabled status condition
• Add PUT response payload to /cluster/* resources
• Normalize backup before performing access check
• Resource state enhancements
• Remove static securityContext
• Suppress serialization of empty labels, properties
• Pin Helm chart latest version
• Make JSON serialization backwards compatible
• Allow user to be updated to add cross organization access
• Added a configMap for mapping SLA to Helm chart version
• Add helm support for token authentication
• Disregard controller update errors for VolumeSnapshots
• Return created/updated resource content for PUT requests

NuoDB Control Plane 2.4.1

Changelog

• Retry backup hooks on failure
• Trigger backup reconciliation on operator restart
• Fixing various backup issues
• Add --restore-from-backup to 'nuodb-cp database create'
• Suppress serialization of empty labels, properties
• Normalize backup before performing access check
• Remove static securityContext
• Make JSON serialization backwards compatible
• Allow user to be updated to add cross organization access
• Disregard controller update errors for VolumeSnapshots
• Update version to 2.4.1

NuoDB Control Plane 2.4.0

Changelog

New features in this release

• Snapshot-based backup for Database custom resources has been added to the Operator.
• The /backups REST resource has been added which allows databases to be backed up via the REST API.
• The nuodb-cp backups subcommand has been added to manage backups.
• The ability to restore/clone databases from backups has been added to the Operator and the REST API.
• TLS secrets for the Domain custom resource are now generated by the Operator, which performs renewal at a configurable time threshold before expiration.
• Token-based (Bearer) authentication scheme has been added that allows time-bound and access-restricted tokens to be generated by users.

Changes in this release

• Import database backup from catalog
• Fix backup stuck in deletion
• Migrate operator to kubebuilder layout v4
• Remove required flag from dbaPassword property
• DatabaseBackup validation
• Ignore conflict on failed release reconciliation
• Allow waiting for volume snapshots to be ready
• Report database in state Restoring
• Implement password rotation in the database controller
• Added rotate database password CLI command
• Preserve query parameters when doing path rewrite
• add authorization token implementation
• Add extensions to OpenAPI properties exposed as Terraform attributes
• Add simulated reconciliation, package test helper
• Enable backup and restore by default
• Enable database backup with external journal
• Added support for database clone
• Add nuodb-cp backup subcommand
• [DBAAS-231] Token Authentication
• Add restoreFrom to database model
• Add controller in the backup manager
• Fix nuodb-cp generate doc formatting
• Use /backups REST API in end-to-end test
• Add /backups REST resource
• Added Embedded backup manager
• Reconcile DatabaseBackup resource
• Added Backup API
• Enable webhooks by default if cert-manager is available
• Resource stuck in deleting due to finalizer not removed
• Manage TLS secret lifecycle with the operator
• Quote hex-encoded SHA for ConfigMap

NuoDB Control Plane 2.3.2

Changelog

• Preserve query parameters when doing path rewrite

NuoDB Control Plane 2.3.1

Changelog

• Quote hex-encoded SHA for ConfigMap

NuoDB Control Plane 2.3.0

Changelog

New features in this release

• Helm features can now be injected with values specified on project and database resources using the tierParameters property.
• The version of NuoDB to use for a project or database can now be specified using the productVersion property.
• Labels can now be attached to project, database, and user resources and filters can be specified based on labels when listing resources using GET.
• Resources can now be listed at higher levels of the resource hierarchy.
  For example, all projects across all organizations or all databases within an organization can be listed.
  The listAccessible query parameter enables discovery of resources accessible to the current user.
• Blocking deletion of resources has been added to the REST API and resource dependencies are now managed using owner references so that deletion of resources using kubectl delete does not leave behind ancillary resources.
• Lifecycle states have been added for project and database resources that summarize the state of the resource, e.g. Creating, Modifying, Available.
• REST endpoints have been added for resources managed by cluster administrators; database quotas, Helm features, and service tiers.
• Automatic renewal of TLS certificates generated by the DBaaS operator for Domain resources has been added.

Changes in this release

• Explicitly inherit database tier parameters from project
• Expose image version via REST API
• Generate documentation for nuodb-cp CLI
• Add --timeout to all delete subcommands
• Use project's service tier parameters for databases by default
• Add user-defined labels for filtering of resources
• Upgrade JDBC driver and add end-to-end testing of nuodb-cp
• Only check explicit volume sizes in REST service
• Update owner once dependents have been created
• Expose timeout to enable blocking delete
• Manage groups of resources using owner references
• Empty ConfigMap reconciliation request for injector
• Expose discrete project and database state
• Allow accessible resources to be discovered
• Support for HelmFeature parameters
• Create /cluster REST resources to manage cluster config
• Added NetworkPolicy resources
• Allow DBaaS resources to be listed at cluster scope
• Automatically renew self-signed TLS certificates for Domain
• Changes in immutable label selector fails Helm upgrade

NuoDB REST Service Test Helper

REST service test helper

This is a package that includes a development version of the REST service JAR that includes support for running a CRUD-only Kubernetes API server, along with a mock controller that simulates reconciliation behavior of the NuoDB DBaaS Operator.

To use this, download the attached test-helper.tgz artifact, unpack, and run the contained setup-rest.sh script:

tar -xf test-helper.tgz
./test-helper/setup-rest.sh

This will create a CRUD-only Kubernetes environment consisting of a Kubernetes API server and etcd (see envtest), and the NuoDB REST service configured to use the Kubernetes environment.

To enable usage of kubectl and nuodb-cp with this environment, run the following:

export KUBECONFIG="$(pwd)/test-helper/kubeconfig.yml"
alias nuodb-cp="java -jar $(pwd)/test-helper/nuodb-control-plane.jar"
eval "$(nuodb-cp generate-completion)"

Mock reconciliation

To enable the mock reconciliation controller, the servers can be started with the MARK_AS_READY=true enviroment variable:

MARK_AS_READY=true ./test-helper/setup-rest.sh

This causes a Kubernetes configmap to be created that enables reconciliation of Domain and Database custom resources, which has the following format:

apiVersion: v1
kind: ConfigMap
metadata:
  name: mock-operator-policy
data:
  markAsReady: "true"
  readinessDelaySeconds: ""

This configmap can also be created or modified while the servers are running to change the reconciliation behavior. The markAsReady key enables reconciliation behavior, while the readinessDelaySeconds key insert the specified delay before making a status update to any resource.

NOTE: The quotes for the values are important, because the configmap schema only accepts string values in the data fields. readinessDelaySeconds: "5" is correct, while readinessDelaySeconds: 5 will be rejected by the API server.

Using nuodb-cp

Following the instructions above, nuodb-cp should be usable without setting up any authentication credentials, since the server is running outside of the Kubernetes cluster and is configured to bypass authentication for client connections coming from the same host.

The REST server will be listening on port 8080 and it is not required to include any path prefix when sending requests to it, because there is no Ingress controller dispatching requests to it based on path prefix rules. The default --url-base value of http://localhost:8080 should just work.

The following commands can be run to create a project and database (which do not have any actual process resources associated with them):

nuodb-cp project create org/proj --sla dev --tier n0.nano
nuodb-cp database create org/proj/db --dba-password db

With the mock reconciler enabled, the project and database will immediately show as Available (or with some delay if readinessDelaySeconds is configured):

$ nuodb-cp project get org/proj
{
  "organization": "org",
  "name": "proj",
  "labels": { },
  "sla": "dev",
  "tier": "n0.nano",
  "properties": {
    "tierParameters": { },
    "productVersion": "5.0"
  },
  "resourceVersion": "1377",
  "status": {
    "caPem": "-----BEGIN CERTIFICATE-----\nMIICwzCCAaugAwIBAgIIJWvF7FHoUL0wDQYJKoZIhvcNAQELBQAwFzEVMBMGA1UE\nAwwMY2EubnVvZGIuY29tMB4XDTI0MDIwMzA0NDgwMVoXDTI1MDIwMjA0NDgwMVow\nFzEVMBMGA1UEAwwMY2EubnVvZGIuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A\nMIIBCgKCAQEArRJHuBmBn9d7PfAmFwHwIxKj69GtbPJNOiySU18En4dTVF0pLaoJ\nBEo+mebMLkQhcrU87A+RqpPNLPLiPvujysXoqNUGzNsDLMMsthe3NSwHYI1u+Vcl\nrWDds+AQvgXzk+/KoInCvAxcGp0zgKv0qje/Bf0Io9Q1VgtfLUvgCea+cSN8j+Ty\n+xzFcHAAFGIkQ0DtW5qnhO6Ni4n+ftfgQOyclahQmWUlsP/5woFjZMN4yc3ZVmuE\njxTc4NmgCOBkMDwBGP0NClEQWdAEPRWXGOoQKnqCD3xcfogwdxniy8IJ779HH+VM\n3ufj59lB01yHbR342QB4mmR4Yvb0m6Su/wIDAQABoxMwETAPBgNVHRMBAf8EBTAD\nAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQCGOckt0j6a9dQWpspZG5dh9rXP/5EQPDrZ\nagvpdTKmK2ZX9cyV/YT4qayXzNlBsZjR8zuohwKWUVGvfdFGSwgezG//Z9B4ZCgY\nopNUhKRS4j6ctg3bIEmbx8Odf7mnLR/hXdhLjo6wUDkow6aFnzd0cn0f6HQJM7Gq\nJWAS1A784NSXyyMeXZuP2SmrVyM8nzT8h7ak1m9ZpiCnDTqCnSyHL5nLtuhcKWDP\nTYMQhs5Wk4ZPYkxWa7Qvk+ClqpzzJ2HYoZJxjQ4eTST1RMu7pNXUcm2MnNbSHnlz\n3bSZjxbX5lkbCqtvyejNAFYSbYGSuGjoywppLmZ8fbVMJAnv4oQ7\n-----END CERTIFICATE-----",
    "ready": true,
    "shutdown": false,
    "message": "Resource is ready",
    "state": "Available",
    "conditions": [ {
      "lastTransitionTime": "2024-02-03T04:48:06Z",
      "message": "Resource is ready",
      "reason": "ReconciliationSucceeded",
      "status": "True",
      "type": "Ready"
    } ]
  }
}

$ nuodb-cp database get org/proj/db
{
  "organization": "org",
  "project": "proj",
  "name": "db",
  "labels": { },
  "tier": "n0.nano",
  "properties": {
    "tierParameters": { },
    "productVersion": "5.0"
  },
  "resourceVersion": "1387",
  "status": {
    "sqlEndpoint": "proj-f4ccefc29c8d.it.nuodb.local",
    "caPem": "-----BEGIN CERTIFICATE-----\nMIICwzCCAaugAwIBAgIIJWvF7FHoUL0wDQYJKoZIhvcNAQELBQAwFzEVMBMGA1UE\nAwwMY2EubnVvZGIuY29tMB4XDTI0MDIwMzA0NDgwMVoXDTI1MDIwMjA0NDgwMVow\nFzEVMBMGA1UEAwwMY2EubnVvZGIuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A\nMIIBCgKCAQEArRJHuBmBn9d7PfAmFwHwIxKj69GtbPJNOiySU18En4dTVF0pLaoJ\nBEo+mebMLkQhcrU87A+RqpPNLPLiPvujysXoqNUGzNsDLMMsthe3NSwHYI1u+Vcl\nrWDds+AQvgXzk+/KoInCvAxcGp0zgKv0qje/Bf0Io9Q1VgtfLUvgCea+cSN8j+Ty\n+xzFcHAAFGIkQ0DtW5qnhO6Ni4n+ftfgQOyclahQmWUlsP/5woFjZMN4yc3ZVmuE\njxTc4NmgCOBkMDwBGP0NClEQWdAEPRWXGOoQKnqCD3xcfogwdxniy8IJ779HH+VM\n3ufj59lB01yHbR342QB4mmR4Yvb0m6Su/wIDAQABoxMwETAPBgNVHRMBAf8EBTAD\nAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQCGOckt0j6a9dQWpspZG5dh9rXP/5EQPDrZ\nagvpdTKmK2ZX9cyV/YT4qayXzNlBsZjR8zuohwKWUVGvfdFGSwgezG//Z9B4ZCgY\nopNUhKRS4j6ctg3bIEmbx8Odf7mnLR/hXdhLjo6wUDkow6aFnzd0cn0f6HQJM7Gq\nJWAS1A784NSXyyMeXZuP2SmrVyM8nzT8h7ak1m9ZpiCnDTqCnSyHL5nLtuhcKWDP\nTYMQhs5Wk4ZPYkxWa7Qvk+ClqpzzJ2HYoZJxjQ4eTST1RMu7pNXUcm2MnNbSHnlz\n3bSZjxbX5lkbCqtvyejNAFYSbYGSuGjoywppLmZ8fbVMJAnv4oQ7\n-----END CERTIFICATE-----",
    "ready": true,
    "shutdown": false,
    "message": "Resource is ready",
    "state": "Available",
    "conditions": [ {
      "lastTransitionTime": "2024-02-03T04:48:37Z",
      "message": "Resource is ready",
      "reason": "ReconciliationSucceeded",
      "status": "True",
      "type": "Ready"
    } ]
  }
}

Shutting down the servers

To shutdown the servers, run the teardown-rest.sh script from the package.

./test-helper/teardown-rest.sh