feat(rds): enhanced monitoring configuration at the cluster level #32157

badmintoncryer · 2024-11-16T06:30:18Z

Issue # (if applicable)

None

Reason for this change

AWS RDS now supports the configuration of advanced monitoring at the cluster level.

https://aws.amazon.com/about-aws/whats-new/2024/09/amazon-rds-performance-insights-supports-aurora-cluster-level-configuration/?nc1=h_ls

This feature is essential for creating limitless database cluster. (#32151)

Description of changes

I added enableClusterLevelEnhancedMonitoring prop to ClusterProps.
If this prop enabled, monitoringInterval and monitoringRole is applied to not the instances but the cluster.

Enhanced monitoring configuration for the cluster and instances has some restrictions.

When setting the monitoringRoleArn for both the cluster and instances, the values must be identical.
When setting the monitoringInterval for both the cluster and instances, the values must be identical.
When the monitoringInterval or monitoringRoleArn is set at the cluster level, enhanced monitoring will be enabled even if the same values are not specified for the instances.

Based on the above, I decided to add a flag (enableClusterLevelEnhancedMonitoring) to switch the application of monitoringInterval and monitoringRole between the instance level and the cluster level to avoid breaking changes to the existing props.

Verification Results Memo

The value of the Enhanced Monitoring configuration must be the same for both the cluster level and the instance level.

  "DatabaseB269D8BB": {
   "Type": "AWS::RDS::DBCluster",
   "Properties": {
    ...,
    "Engine": "aurora-postgresql",
    "EngineVersion": "16.1",
    "MonitoringInterval": 5,
    "MonitoringRoleArn": {
     "Fn::GetAtt": [
      "DatabaseMonitoringRoleForCluster91F5067E",
      "Arn"
     ]
    },
  },
  "DatabasewriterInstanceEBFCC003": {
   "Type": "AWS::RDS::DBInstance",
   "Properties": {
    ...,
    "DBClusterIdentifier": {
     "Ref": "DatabaseB269D8BB"
    },
    "Engine": "aurora-postgresql",
    "MonitoringInterval": 1,
    "MonitoringRoleArn": {
     "Fn::GetAtt": [
      "DatabaseMonitoringRole576991DA",
      "Arn"
     ]
    },
  }

           | Database/writerInstance (DatabasewriterInstanceEBFCC003) Resource handler returned message: "MonitoringInterval conflicts with cluster level parameter. (Service: Rds, Status Code: 400, Request ID: 7c9d9023-fe2a-4b39-a939-22cec3595041)" (RequestToken: bc3e18eb-2daa-4f25-c205-11f1ec70fc15, HandlerErrorCode: InvalidRequest)

The monitoring role arn must be the same for both cluster and instance level.

 "DatabaseB269D8BB": {
   "Type": "AWS::RDS::DBCluster",
   "Properties": {
    ...,
    "Engine": "aurora-postgresql",
    "EngineVersion": "16.1",
    "MonitoringInterval": 5,
    "MonitoringRoleArn": {
     "Fn::GetAtt": [
      "DatabaseMonitoringRoleForCluster91F5067E",
      "Arn"
     ]
    },
  },
  "DatabasewriterInstanceEBFCC003": {
   "Type": "AWS::RDS::DBInstance",
   "Properties": {
    ...,
    "DBClusterIdentifier": {
     "Ref": "DatabaseB269D8BB"
    },
    "Engine": "aurora-postgresql",
    "MonitoringInterval": 5,
    "MonitoringRoleArn": {
     "Fn::GetAtt": [
      "DatabaseMonitoringRole576991DA",
      "Arn"
     ]
    },
  }

Resource handler returned message: "MonitoringRoleArn conflicts with cluster level parameter. (Service: Rds, Status Code: 400, Request ID: 761a867d-409e-4208-a08e-5591607d0ed0)" (RequestToken: fb691324-f37c-405d-e5bd-447911f7664f, HandlerErrorCode: InvalidRequest)

Description of how you validated changes

Add both unit and integ tests

Checklist

My code adheres to the CONTRIBUTING GUIDE and DESIGN GUIDELINES

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

aws-cdk-automation

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

codecov · 2024-11-16T06:41:58Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.17%. Comparing base (b3e39a9) to head (23078ed).
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #32157   +/-   ##
=======================================
  Coverage   77.17%   77.17%           
=======================================
  Files         105      105           
  Lines        7169     7169           
  Branches     1315     1315           
=======================================
  Hits         5533     5533           
  Misses       1455     1455           
  Partials      181      181

Flag	Coverage Δ
suite.unit	`77.17% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
packages/aws-cdk	`77.17% <ø> (ø)`

---- 🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests
JS Bundle Analysis - Avoid shipping oversized bundles

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

GavinZZ

Left some feedback, thanks for the contribution!

GavinZZ · 2024-11-21T19:57:16Z

packages/aws-cdk-lib/aws-rds/README.md

+new rds.DatabaseCluster(this, 'Cluster', {
+  engine: rds.DatabaseClusterEngine.auroraPostgres({ version: rds.AuroraPostgresEngineVersion.VER_16_1 }),
+  writer: rds.ClusterInstance.serverlessV2('writerInstance'),
+  vpc,
+  monitoringInterval: Duration.seconds(5),
+});


I didn't see enableClusterLevelEnhancedMonitoring property set in this codeblock?

This code block is an example of configuring enhanced monitoring at the instance level.

// Enable Enhanced Monitoring at instance level to all instances in the cluster new rds.DatabaseCluster(this, 'Cluster', { engine: rds.DatabaseClusterEngine.auroraPostgres({ version: rds.AuroraPostgresEngineVersion.VER_16_1 }), writer: rds.ClusterInstance.serverlessV2('writerInstance'), vpc, monitoringInterval: Duration.seconds(5), }); // Enable Enhanced Monitoring at the cluster level new rds.DatabaseCluster(this, 'Cluster', { engine: rds.DatabaseClusterEngine.auroraPostgres({ version: rds.AuroraPostgresEngineVersion.VER_16_1 }), writer: rds.ClusterInstance.serverlessV2('writerInstance'), vpc, monitoringInterval: Duration.seconds(5), enableClusterLevelEnhancedMonitoring: true, });

I see! The text above got me confused If you want to enable enhanced monitoring at the cluster level and I didn't notice the comment you. Looks good.

No problem! Thank you for your confirmation!

GavinZZ · 2024-11-21T21:09:42Z

packages/aws-cdk-lib/aws-rds/lib/cluster.ts

+      monitoringInterval: props.enableClusterLevelEnhancedMonitoring ? undefined : props.monitoringInterval,
+      monitoringRole: props.enableClusterLevelEnhancedMonitoring ? undefined : this.monitoringRole,


Would be nice to have a comment to explain that when enableClusterLevelEnhancedMonitoring is enabled, we already set interval and role at cluster level so no need to re-set it in instance level.

I have added such explanations to all the sections where instance-level enhanced monitoring settings are applied.

GavinZZ · 2024-11-21T21:13:54Z

...eg/test/aws-rds/test/integ.temp.js.snapshot/EnhancedMonitoringClusterTestStack.template.json

+{
+ "Resources": {
+  "Vpc8378EB38": {
+   "Type": "AWS::EC2::VPC",
+   "Properties": {
+    "CidrBlock": "10.0.0.0/16",
+    "EnableDnsHostnames": true,
+    "EnableDnsSupport": true,
+    "InstanceTenancy": "default",
+    "Tags": [
+     {
+      "Key": "Name",
+      "Value": "EnhancedMonitoringClusterTestStack/Vpc"
+     }
+    ]


what is this integ.temp.js test? I don't see any integration testss attach to it.

I'm sorry.. I've removed it.

packages/aws-cdk-lib/aws-rds/README.md

Co-authored-by: GZ <[email protected]>

Pull request has been modified.

Co-authored-by: GZ <[email protected]>

aws-cdk-automation · 2024-11-22T03:25:52Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
Commit ID: 23078ed
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mergify · 2024-11-22T03:26:25Z

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

github-actions · 2024-11-22T03:26:46Z

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

badmintoncryer added 2 commits November 16, 2024 00:25

add integ test

d99e8b0

Merge remote-tracking branch 'origin/main' into enhanced-monitoring

e128357

aws-cdk-automation requested a review from a team November 16, 2024 06:30

github-actions bot added p2 distinguished-contributor [Pilot] contributed 50+ PRs to the CDK labels Nov 16, 2024

aws-cdk-automation previously requested changes Nov 16, 2024

View reviewed changes

badmintoncryer added 3 commits November 16, 2024 21:28

rename parameter

fd1c170

add validation

c02d340

update readme

1d1b8a8

update unit test

26f4829

badmintoncryer marked this pull request as ready for review November 16, 2024 13:58

badmintoncryer changed the title ~~feat(rds): enhanced monitoring configuration for cluster level~~ feat(rds): enhanced monitoring configuration for the cluster level Nov 16, 2024

badmintoncryer changed the title ~~feat(rds): enhanced monitoring configuration for the cluster level~~ feat(rds): enhanced monitoring configuration at the cluster level Nov 16, 2024

badmintoncryer added 5 commits November 17, 2024 21:26

update readme

828ddf7

Merge branch 'main' into enhanced-monitoring

cd93d26

update readme

be282da

update readme

c974f6f

Merge branch 'main' into enhanced-monitoring

a85a5cf

badmintoncryer mentioned this pull request Nov 18, 2024

feat(rds): limitless database cluster #32151

Merged

1 task

aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Nov 18, 2024

GavinZZ previously requested changes Nov 21, 2024

View reviewed changes

aws-cdk-automation removed the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Nov 21, 2024

Update packages/aws-cdk-lib/aws-rds/README.md

ce187c6

Co-authored-by: GZ <[email protected]>

Update packages/aws-cdk-lib/aws-rds/README.md

f79868f

Co-authored-by: GZ <[email protected]>

aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Nov 22, 2024

badmintoncryer added 2 commits November 22, 2024 11:52

update

4ab8b76

Merge branch 'main' into enhanced-monitoring

23078ed

GavinZZ approved these changes Nov 22, 2024

View reviewed changes

mergify bot merged commit 01f2dcd into aws:main Nov 22, 2024
20 checks passed

github-actions bot locked as resolved and limited conversation to collaborators Nov 22, 2024

badmintoncryer deleted the enhanced-monitoring branch November 22, 2024 03:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(rds): enhanced monitoring configuration at the cluster level #32157

feat(rds): enhanced monitoring configuration at the cluster level #32157

badmintoncryer commented Nov 16, 2024 •

edited

Loading

aws-cdk-automation left a comment

codecov bot commented Nov 16, 2024 •

edited

Loading

GavinZZ left a comment

GavinZZ Nov 21, 2024

badmintoncryer Nov 22, 2024

GavinZZ Nov 22, 2024

badmintoncryer Nov 22, 2024

GavinZZ Nov 21, 2024

badmintoncryer Nov 22, 2024

GavinZZ Nov 21, 2024

badmintoncryer Nov 22, 2024

aws-cdk-automation commented Nov 22, 2024

mergify bot commented Nov 22, 2024

github-actions bot commented Nov 22, 2024

		monitoringInterval: props.enableClusterLevelEnhancedMonitoring ? undefined : props.monitoringInterval,
		monitoringRole: props.enableClusterLevelEnhancedMonitoring ? undefined : this.monitoringRole,

feat(rds): enhanced monitoring configuration at the cluster level #32157

feat(rds): enhanced monitoring configuration at the cluster level #32157

Conversation

badmintoncryer commented Nov 16, 2024 • edited Loading

Issue # (if applicable)

Reason for this change

Description of changes

Verification Results Memo

Description of how you validated changes

Checklist

aws-cdk-automation left a comment

Choose a reason for hiding this comment

codecov bot commented Nov 16, 2024 • edited Loading

Codecov Report

GavinZZ left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aws-cdk-automation commented Nov 22, 2024

AWS CodeBuild CI Report

mergify bot commented Nov 22, 2024

github-actions bot commented Nov 22, 2024

badmintoncryer commented Nov 16, 2024 •

edited

Loading

codecov bot commented Nov 16, 2024 •

edited

Loading