Expose CloudFormation Custom Resource Emulator Resource #1807

flostadler · 2024-11-07T16:50:54Z

This change exposes the new CloudFormation Custom Resource Emulator resource.
Additionally, it adds an integration test for it and makes Check correctly handle unknowns.

One follow up item is to translate the code example to other languages.

flostadler · 2024-11-07T16:50:56Z

This change is part of the following stack:

Expose CloudFormation Custom Resource Emulator Resource #1807 ◀

_{Change managed by git-spice.}

github-actions · 2024-11-07T16:52:40Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

cloudformation.CustomResourceEmulator

codecov · 2024-11-08T10:23:06Z

Codecov Report

Attention: Patch coverage is 94.73684% with 1 line in your changes missing coverage. Please review.

Project coverage is 49.65%. Comparing base (14a21ae) to head (f0d2ac8).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
provider/pkg/provider/provider.go	85.71%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1807      +/-   ##
==========================================
+ Coverage   48.13%   49.65%   +1.51%     
==========================================
  Files          43       43              
  Lines        6554     6559       +5     
==========================================
+ Hits         3155     3257     +102     
+ Misses       3156     3059      -97     
  Partials      243      243

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

flostadler · 2024-11-08T12:08:32Z

provider/pkg/provider/provider.go

@@ -1125,7 +1135,7 @@ func (p *cfnProvider) Delete(ctx context.Context, req *pulumirpc.DeleteRequest)
 			KeepSecrets:  true,
 		})
 		if err != nil {
-			return nil, errors.Wrapf(err, "failed to parse inputs for update")
+			return nil, errors.Wrapf(err, "failed to parse inputs for delete")


Found this copy&paste error in the message. We're in the delete method here

corymhall · 2024-11-08T14:23:39Z

provider/pkg/schema/docs/content/cfn-custom-resource.md

+
+CloudFormation Custom Resources allow you to write custom provisioning logic for resources that aren't directly available as AWS CloudFormation resource types. Common use cases include:
+
+- Managing resources outside of AWS (e.g., GitHub repositories, external APIs)


Should we remove this one or add a comment saying that it would be better to use one of the other providers instead?

corymhall · 2024-11-08T14:26:07Z

examples/cfn-custom-resource/ami-lookup.js

+
+exports.handler = function(event, context) {
+
+    console.log("REQUEST RECEIVED:\n" + JSON.stringify(event));


We should remove the ResponseURL when logging the event for security reasons.

corymhall · 2024-11-08T14:27:39Z

examples/cfn-custom-resource/index.ts

+    },
+});
+
+const rpa1 = new awsClassic.iam.RolePolicyAttachment("lambdaRolePolicyAttachment1", {


as a side note it's hilarious to me that CCAPI doesn't support these basic iam resources. How are you supposed to do anything in AWS without these 🙃

corymhall · 2024-11-08T14:28:33Z

examples/cfn-custom-resource/index.ts

+
+// Create the Lambda function for the custom resource
+const lambdaFunction = new awsClassic.lambda.Function("ami-lookup-custom-resource", {
+    runtime: awsClassic.types.enums.lambda.Runtime.NodeJS16dX,


Should we use a newer version? This test will probably start failing sooner rather than later if we use this old of a runtime.

corymhall · 2024-11-08T14:29:29Z

examples/cfn-custom-resource/index.ts

+    resourceType: 'Custom::MyResource',
+}, { customTimeouts: { create: '5m', update: '5m', delete: '5m' } });
+
+const cloudformationStack = new awsClassic.cloudformation.Stack('stack', {


t0yv0 · 2024-11-08T15:37:26Z

provider/pkg/schema/docs/content/cfn-custom-resource.md

+    customResourceProperties: {
+        hello: "world"
+    },
+    serviceToken: "arn:aws:lambda:us-west-2:123456789012:function:my-custom-resource",


This lambda would be provisioned by CF, is that right?

No, it is provisioned by the user in their Pulumi program or CloudFormation Stack.

This Example usage should probably have a link to how to do that.

The Lambda Function Requirements section explains how to do that. I don't think we should add a full Custom Resource implementation here. That would essentially be the cfn-custom-resource example I've added (and that's multiple files)

Yes exactly, so perhaps consider adding a link to it?

Yeah, I can add that once it's merged into master :)

t0yv0 · 2024-11-08T15:39:03Z

examples/cfn-custom-resource/index.ts

+    policyArn: awsClassic.iam.ManagedPolicies.AWSLambdaBasicExecutionRole,
+});
+
+const bucket = new awsClassic.s3.Bucket('custom-resource-emulator', {


t0yv0 · 2024-11-08T15:40:04Z

examples/cfn-custom-resource/index.ts

+        Region: amiRegion,
+        Architecture: 'HVM64',
+    },
+    serviceToken: lambdaFunction.arn,


Ahhh so serviceToken is not called lambdaFunctionARN because some custom resources are backed by something else?

Yeah, there's also SNS backed Custom Resources: #1812

They're not widely used, so we decided to skip support for them for now. It's not in scope of the CDK work

The naming makes a bit more sense now.

This PR adds support for [CloudFormation Custom Resource](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources.html) to the aws-native provider. It implements an emulator that enables Pulumi programs to interact with Lambda-backed CloudFormation Custom Resources. A CloudFormation custom resource is essentially an extension point to run arbitrary code as part of the CloudFormation lifecycle. It is similar in concept to the [Pulumi Command Provider](https://www.pulumi.com/registry/packages/command/), the difference being that CloudFormation CustomResources are executed in the Cloud; either through Lambda or SNS. For the first implementation we decided to limit the scope to Lambda backed Custom Resources, because the SNS variants are not widely used. ## Custom Resource Protocol The implementation follows the CloudFormation Custom Resource protocol. I derived the necessary parts by combining information from the [docs](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/crpg-ref.html), [CDKs CustomResource Framework](https://github.com/aws/aws-cdk/tree/main/packages/%40aws-cdk/custom-resource-handlers/lib/custom-resources-framework) and trial&error. Notable aspects of that protocol are: - primitive properties need to be string encoded when sending them to Custom Resource handlers. This includes deeply nested properties: aws-cloudformation/cloudformation-coverage-roadmap#1037 - The Lambda Function is invoked asynchronously. Lambda will retry the execution if the function fails unexpectedly (e.g. unhandled exception). - Due to the async invocation, the response is not returned from the Lambda Function, instead it's sent to a `ResponseURL` that needs to be included in the request payload. - Similarly to CloudFormation, we decided to implement this using S3 Buckets and presigned URLs. ### Custom Resource Lifecycle ```mermaid sequenceDiagram participant A as aws-native participant S3 as S3 Bucket participant L as Lambda %% Create Flow Note over A,L: Create Operation A->>S3: Generate presigned URL A->>L: Invoke with CREATE event activate L loop Until response found or timeout A->>S3: Poll for response L-->>S3: Upload response end deactivate L A->>S3: Fetch response alt Success A->>A: Store PhysicalId & outputs else Failure A->>A: Return error end %% Update Flow Note over A,L: Update Operation A->>S3: Generate presigned URL A->>L: Invoke with UPDATE event activate L loop Until response found or timeout A->>S3: Poll for response L-->>S3: Upload response end deactivate L A->>S3: Fetch response alt Success A->>A: Check PhysicalId alt ID Changed A->>S3: Generate presigned URL for cleanup A->>L: Invoke with DELETE event for old resource activate L loop Until cleanup response found or timeout A->>S3: Poll for cleanup response L-->>S3: Upload cleanup response end deactivate L A->>S3: Fetch cleanup response end else Failure A->>A: Return error end %% Delete Flow Note over A,L: Delete Operation A->>S3: Generate presigned URL A->>L: Invoke with DELETE event activate L loop Until response found or timeout A->>S3: Poll for response L-->>S3: Upload response end deactivate L A->>S3: Fetch response alt Success A->>A: Return success else Failure A->>A: Return error end ``` ## Reviewer Notes Key areas to review: 1. Error handling in the response collection mechanism 2. Timeout management, especially for the `Update` lifecycle 3. Documentation completeness and accuracy Exposing this resource and schematizing it is part of this PR #1807. Automatically cleaning up the response objects is not included in this PR in order to keep its size manageable. Implementing this is tracked here: #1813. Please pay special attention to: - S3 response collection mechanism security - State management during updates - Cleanup handling when physical resource IDs change ## Testing - Unit tests including error handling tests for various failure scenarios - Integration tests with actual Lambda functions are added in this stacked PR: #1807 ## Related Issues - pulumi/pulumi-cdk#109 - #1812 - #1813

examples/examples_nodejs_test.go

flostadler · 2024-11-11T12:47:10Z

provider/pkg/resources/cfn_custom_resource.go

 		// if the stack ID is not provided, we use the pulumi stack ID as the stack ID
 		inputs[resource.PropertyKey("stackId")] = resource.NewStringProperty(urn.Stack().String())
 	}

-	if typedInputs.CustomResourceProperties != nil {


I decided to move stringifying the customResourceProperties into the Create/Update/Delete methods.

This is an implementation detail and shouldn't really be done in Check. It would also complicate handling secrets and unknowns.

t0yv0 · 2024-11-11T14:36:50Z

provider/pkg/resources/cfn_custom_resource_test.go

+			},
+		},
+		{
+			name: "Preserves Secrets",


t0yv0 · 2024-11-11T14:38:14Z

examples/examples_nodejs_test.go

+	previewResult = test.Preview(t)
+	assertpreview.HasNoChanges(t, previewResult)
+
+	test.Destroy(t)


I think this is automatically done and not necessary to call explicitly.

Yeah it is, but I wanted to be explicit about this being one of the tested aspects. Happy to remove it though

t0yv0 · 2024-11-11T14:40:36Z

examples/cfn-custom-resource/ami-lookup.js

+}
+
+// Send response to the pre-signed S3 URL 
+async function sendResponse(event, context, responseStatus, responseData) {


I am curious here, when people write lambdas to power custom resources in CloudFormation, do they need to take care of this functionality in user code or is there some library or framework that users can call in CF?

This needs to be part of the user code in the lambda function.

When using Inline Code in CloudFormation you can use the cfn-response module if it's a node Lambda. That's automatically included in the Lambda Zip when using inline code.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-lambda-function-code-cfnresponsemodule.html#cfn-lambda-function-code-cfnresponsemodule-source

You could use it in the example too presumably? https://www.npmjs.com/package/cfn-response seems available. No big deal though

That's not the official version. Somebody just extracted it (from a lambda function) and published it on Lambda. I wouldn't wanna rely on this in-official version for the example.
Not worth it for a simple http put request IMO

This was referenced Nov 7, 2024

Add AWS SDK clients for CFN Custom Resource support #1805

Merged

Add implementation for CloudFormation Custom Resource Emulator #1806

Merged

flostadler force-pushed the flostadler/cfn-custom-resource-impl branch from baf7ba4 to 1d4b118 Compare November 7, 2024 17:32

flostadler force-pushed the flostadler/cfn-custom-resource-expose branch from adff217 to 8a8c517 Compare November 7, 2024 17:32

flostadler mentioned this pull request Nov 7, 2024

Add utility functions for supporting CloudFormation CustomResources #1808

Merged

flostadler force-pushed the flostadler/cfn-custom-resource-expose branch 2 times, most recently from c222a0e to a78e2d5 Compare November 7, 2024 18:19

flostadler force-pushed the flostadler/cfn-custom-resource-impl branch from 59357d1 to 94cfa06 Compare November 8, 2024 10:19

flostadler force-pushed the flostadler/cfn-custom-resource-expose branch from a78e2d5 to 6464a5f Compare November 8, 2024 10:19

flostadler requested review from t0yv0 and corymhall November 8, 2024 11:57

flostadler self-assigned this Nov 8, 2024

flostadler requested a review from a team November 8, 2024 11:57

flostadler marked this pull request as ready for review November 8, 2024 11:57

flostadler force-pushed the flostadler/cfn-custom-resource-impl branch from 94cfa06 to 19b72f2 Compare November 8, 2024 12:01

flostadler force-pushed the flostadler/cfn-custom-resource-expose branch from 6464a5f to 939b0b3 Compare November 8, 2024 12:01

flostadler commented Nov 8, 2024

View reviewed changes

flostadler force-pushed the flostadler/cfn-custom-resource-impl branch from 19b72f2 to bc9e116 Compare November 8, 2024 13:53

flostadler force-pushed the flostadler/cfn-custom-resource-expose branch from ee9200c to ba6b3ba Compare November 8, 2024 13:53

corymhall approved these changes Nov 8, 2024

View reviewed changes

t0yv0 reviewed Nov 8, 2024

View reviewed changes

t0yv0 approved these changes Nov 8, 2024

View reviewed changes

flostadler force-pushed the flostadler/cfn-custom-resource-expose branch from d64727a to 5001132 Compare November 8, 2024 16:44

flostadler force-pushed the flostadler/cfn-custom-resource-impl branch from cdfdb9b to 914125c Compare November 8, 2024 16:48

Base automatically changed from flostadler/cfn-custom-resource-impl to master November 8, 2024 18:06

flostadler added 11 commits November 8, 2024 19:06

Expose CloudFormation Custom Resource Emulator Resource

c60d8ec

Regenerate Schema

024669f

Regenerate SDKs

b031e32

Add example for using CloudFormation Custom Resource Emulator

29f7d9c

Add docs for custom resource

940e804

Regen schema

ae82c16

Regen SDK

2cc9972

Remove package-lock.json from PR

8c72236

Correct example name

773a5d6

Remove unrelated change

fefca64

Review comments

670e340

flostadler force-pushed the flostadler/cfn-custom-resource-expose branch from 8496d7d to 670e340 Compare November 8, 2024 18:07

flostadler commented Nov 8, 2024

View reviewed changes

examples/examples_nodejs_test.go Show resolved Hide resolved

flostadler added 2 commits November 10, 2024 13:10

Correctly await cfn response in example

38a15c2

Handle unknowns in preview

bdaa07a

flostadler requested review from t0yv0 and corymhall November 11, 2024 11:35

flostadler added 2 commits November 11, 2024 12:42

Make Check handle secrets

438e2aa

Move map stringifying to CRUD methods

fbf3f9f

flostadler commented Nov 11, 2024

View reviewed changes

Add tests for stringifying inputs

f0d2ac8

t0yv0 reviewed Nov 11, 2024

View reviewed changes

provider/pkg/resources/cfn_custom_resource_test.go

},

},

{

name: "Preserves Secrets",

Copy link

Member

t0yv0 Nov 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

t0yv0 reviewed Nov 11, 2024

View reviewed changes

t0yv0 approved these changes Nov 11, 2024

View reviewed changes

flostadler merged commit 7968d1b into master Nov 11, 2024
18 checks passed

flostadler deleted the flostadler/cfn-custom-resource-expose branch November 11, 2024 14:58

flostadler mentioned this pull request Nov 14, 2024

sdk/python/pulumi_aws_native/cloudformation/__init__.py Bug Report #1826

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Expose CloudFormation Custom Resource Emulator Resource #1807

Expose CloudFormation Custom Resource Emulator Resource #1807

flostadler commented Nov 7, 2024 •

edited

Loading

flostadler commented Nov 7, 2024 •

edited

Loading

github-actions bot commented Nov 7, 2024 •

edited

Loading

codecov bot commented Nov 8, 2024 •

edited

Loading

flostadler Nov 8, 2024

corymhall Nov 8, 2024

corymhall Nov 8, 2024

corymhall Nov 8, 2024

corymhall Nov 8, 2024

corymhall Nov 8, 2024

t0yv0 Nov 8, 2024

flostadler Nov 8, 2024

t0yv0 Nov 8, 2024

flostadler Nov 8, 2024

t0yv0 Nov 11, 2024

flostadler Nov 11, 2024

t0yv0 Nov 8, 2024

t0yv0 Nov 8, 2024

flostadler Nov 8, 2024

t0yv0 Nov 8, 2024

flostadler Nov 11, 2024 •

edited

Loading

t0yv0 Nov 11, 2024

t0yv0 Nov 11, 2024

flostadler Nov 11, 2024

t0yv0 Nov 11, 2024

flostadler Nov 11, 2024

t0yv0 Nov 11, 2024

flostadler Nov 11, 2024 •

edited

Loading


		CloudFormation Custom Resources allow you to write custom provisioning logic for resources that aren't directly available as AWS CloudFormation resource types. Common use cases include:

		- Managing resources outside of AWS (e.g., GitHub repositories, external APIs)


		exports.handler = function(event, context) {

		console.log("REQUEST RECEIVED:\n" + JSON.stringify(event));

Expose CloudFormation Custom Resource Emulator Resource #1807

Expose CloudFormation Custom Resource Emulator Resource #1807

Conversation

flostadler commented Nov 7, 2024 • edited Loading

flostadler commented Nov 7, 2024 • edited Loading

github-actions bot commented Nov 7, 2024 • edited Loading

Does the PR have any schema changes?

New resources:

codecov bot commented Nov 8, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

flostadler Nov 11, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

flostadler Nov 11, 2024 • edited Loading

Choose a reason for hiding this comment

flostadler commented Nov 7, 2024 •

edited

Loading

flostadler commented Nov 7, 2024 •

edited

Loading

github-actions bot commented Nov 7, 2024 •

edited

Loading

codecov bot commented Nov 8, 2024 •

edited

Loading

flostadler Nov 11, 2024 •

edited

Loading

flostadler Nov 11, 2024 •

edited

Loading