Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Pattern: API Gateway Save to DynamoDB and Stream to Lambda with CDK which combine both TypeScript for CDK and Python Lambda Code #1519

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions apigw-rest-api-dynamodb-lambda-cdk/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
*.js
!jest.config.js
*.d.ts
node_modules

# CDK asset staging directory
.cdk.staging
cdk.out
6 changes: 6 additions & 0 deletions apigw-rest-api-dynamodb-lambda-cdk/.npmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
*.ts
!*.d.ts

# CDK asset staging directory
.cdk.staging
cdk.out
158 changes: 158 additions & 0 deletions apigw-rest-api-dynamodb-lambda-cdk/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
# Amazon API Gateway REST API to Amazon DynamoDB

This pattern creates an Amazon API Gateway REST API that integrates with an Amazon DynamoDB table.

Learn more about this pattern at Serverless Land Patterns: http://serverlessland.com/patterns/apigw-dynamodb-lambda-cdk.

Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.

## Requirements

* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
* [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) (AWS SAM) installed

## Architecture
The following diagram illustrates the solutions architect
![Architecture Diagram](img/concept.png)
## Deployment Instructions

1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository:
```
git clone https://github.com/aws-samples/serverless-patterns
```
1. Change directory to the pattern directory:
```
cd apigw-rest-api-dynamodb-lambda-cdk
```

1. Install dependencies
```
npm install
cd src
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be cd ..\src

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, I am not sure about this. It should be: cd src

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, just cd src . I tested on Mac

pip install aws-xray-sdk --target .
pip install aws_lambda_powertools --target .
```

1. Deploy the stack to your default AWS account and region. The output of this command should give you the HTTP API URL.
```
nguyendinhthi0705 marked this conversation as resolved.
Show resolved Hide resolved
cd ..
cdk deploy
```

## How it works

This pattern creates an Amazon API Gateway REST API that integrates with an Amazon DynamoDB table. The API integrates directly with the DynamoDB API and supports [PutItem](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_PutItem.html) and [Query](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html) actions. The API Integration has read and write access to the DynamoDB. There is no authentication on the API endpoint

## Testing

Upon deployment, you will see the API endpoint URL in the output. It will take the format:

`https://${API_ID}.execute-api.${REGION_NAME}.amazonaws.com`

Once the application is deployed, use [Postman](https://www.postman.com/) to test the API using the following instructions.

1. Launch Postman

1. Invoke the DynamoDB **PutItem** action to add a new item to the DynamoDB table:
* Enter the API URL with the **prod** stage as the path:.
```
https://${API_ID}.execute-api.${REGION_NAME}.amazonaws.com/prod/awsomedynamodb
```
* Select **POST** as the HTTP method from the drop-down list to the left of the address bar.
* Choose the **Body** tab. Choose **raw** and select **JSON** from the drop-down list. Enter the following into the text box:
```
{
"id":"14",
"name":"Do some thing",
"description":"Do some thing",
"customer":"John"
}
Comment on lines +67 to +71
Copy link
Contributor

@julianwood julianwood Aug 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The items you are adding don't seem relevant to what you query in the testing

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure about this, you can put anything for testing, right ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you can put anything. But although DynamoDB support schemeless, we actually should control scheme in real product

```
* Choose **Send** to submit the request and receive a "200 OK" response.
* Open the DynamoDB console and select the table which was created to confirm that the item has been added.
* Change the values for `pk` or `data` in the POST body and repeat this process to add multiple items to the DynamoDB table.

1. Invoke the DynamoDB **Query** action to query all items by artist in the DynamoDB table:
* Enter the Invoke URL in the address bar. Add **/prod/foo** to the URL path.
* Add **/foo** to the URL path. This defines the ID that you want to query.
```
https://${API_ID}.execute-api.${REGION_NAME}.amazonaws.com/prod/awsomedynamodb
```
* Select **GET** as the HTTP method from the drop-down list to the left of the address bar.
* Choose the **Body** tab. Choose **none**.
* Choose **Send** to submit the request and receive a "200 OK" response with a list of the matching results. Example:
```
{
"Count": 1,
"Items": [
{
"AWSomeDynamoDBId": {
"S": "foo"
},
"name": {
"S": "blah blah blah"
},
"description": {
"S": "blah blah blah"
},
"Id": {
"S": "string"
}
}
],
"ScannedCount": 1
}
```

1. Invoke the DynamoDB **Query** action to query specific item by artist in the DynamoDB table:
* Enter the Invoke URL in the address bar. Add **/prod/foo** to the URL path.
* Add **/foo** to the URL path. This defines the ID that you want to query.
```
https://${API_ID}.execute-api.${REGION_NAME}.amazonaws.com/prod/awsomedynamodb/{id}
```
* Select **GET** as the HTTP method from the drop-down list to the left of the address bar.
* Choose the **Body** tab. Choose **none**.
* Choose **Send** to submit the request and receive a "200 OK" response with a list of the matching results. Example:
```
{
"Count": x,
"Items": [
{
"AWSomeDynamoDBId": {
"S": "foo"
},
"name": {
"S": "blah blah blah"
},
"description": {
"S": "blah blah blah"
},
"Id": {
"S": "string"
}
}
],
"ScannedCount": 1
}
```
## Documentation
- [Tutorial: Build an API Gateway REST API with AWS integration](https://docs.aws.amazon.com/apigateway/latest/developerguide/getting-started-aws-proxy.html)
- [How do I use API Gateway as a proxy for another AWS service?](https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-proxy-integrate-service/)
- [Using Amazon API Gateway as a proxy for DynamoDB](https://aws.amazon.com/blogs/compute/using-amazon-api-gateway-as-a-proxy-for-dynamodb/)
- [Setting up data transformations for REST APIs](https://docs.aws.amazon.com/apigateway/latest/developerguide/rest-api-data-transformations.html)
- [Amazon API Gateway API request and response data mapping reference](https://docs.aws.amazon.com/apigateway/latest/developerguide/request-response-data-mappings.html)
- [API Gateway mapping template and access logging variable reference](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-mapping-template-reference.html)

## Cleanup

Run the given command to delete the resources that were created. It might take some time for the CloudFormation stack to get deleted.
```
cdk destroy
```

----
Copyright 2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.

SPDX-License-Identifier: MIT-0
21 changes: 21 additions & 0 deletions apigw-rest-api-dynamodb-lambda-cdk/bin/ApiDynamoDB.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import { ApiDynamoDBStack } from '../lib/ApiDynamoDBStack';

const app = new cdk.App();
new ApiDynamoDBStack(app, 'ApiDynamoDBStack', {
/* If you don't specify 'env', this stack will be environment-agnostic.
* Account/Region-dependent features and context lookups will not work,
* but a single synthesized template can be deployed anywhere. */

/* Uncomment the next line to specialize this stack for the AWS Account
* and Region that are implied by the current CLI configuration. */
// env: { account: process.env.CDK_DEFAULT_ACCOUNT, region: process.env.CDK_DEFAULT_REGION },

/* Uncomment the next line if you know exactly what Account and Region you
* want to deploy the stack to. */
// env: { account: '123456789012', region: 'us-east-1' },

/* For more information, see https://docs.aws.amazon.com/cdk/latest/guide/environments.html */
});
39 changes: 39 additions & 0 deletions apigw-rest-api-dynamodb-lambda-cdk/cdk.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
{
"app": "npx ts-node --prefer-ts-exts bin/ApiDynamoDB.ts",
"watch": {
"include": [
"**"
],
"exclude": [
"README.md",
"cdk*.json",
"**/*.d.ts",
"**/*.js",
"tsconfig.json",
"package*.json",
"yarn.lock",
"node_modules",
"test"
]
},
"context": {
"@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId": true,
"@aws-cdk/core:stackRelativeExports": true,
"@aws-cdk/aws-rds:lowercaseDbIdentifier": true,
"@aws-cdk/aws-lambda:recognizeVersionProps": true,
"@aws-cdk/aws-lambda:recognizeLayerVersion": true,
"@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021": true,
"@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
"@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
"@aws-cdk/core:checkSecretUsage": true,
"@aws-cdk/aws-iam:minimizePolicies": true,
"@aws-cdk/core:validateSnapshotRemovalPolicy": true,
"@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
"@aws-cdk/aws-s3:createDefaultLoggingPolicy": true,
"@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true,
"@aws-cdk/core:target-partitions": [
"aws",
"aws-cn"
]
}
}
61 changes: 61 additions & 0 deletions apigw-rest-api-dynamodb-lambda-cdk/example-pattern.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
{
"title": "Amazon API Gateway to Amazon DynamoDB and Trigger Event to AWS Lambda as a consumer",
"description": "Create a REST API Gateway which performs PutRecords API to save to DynamoDB and the stream is consumed by a Lambda function",
"language": "Python",
"level": "300",
"framework": "SAM",
"introBox": {
"headline": "How it works",
"text": [
"This pattern will help you to deploy Amazon API Gateway API integration with Amazon DynamoDB and it will deploy Lambda function as a consumer for deployed Amazon DynamoDB Stream.",
"In this pattern, Lambda function is logging the decoded message which was received from the Amazon Kinesis Stream."
]
},
"gitHub": {
"template": {
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/apigw-rest-dynamodb-lambda-cdk",
"templateURL": "serverless-patterns/apigw-rest-dynamodb-lambda-cdk",
"projectFolder": "apigw-rest-dynamodb-lambda-cdk",
"templateFile": "apigw-rest-dynamodb-lambda-cdk/template.yaml"
}
},
"resources": {
"bullets": [
{
"text": "Tutorial: Create a REST API as an Amazon Kinesis proxy in API Gateway",
"link": "https://docs.aws.amazon.com/apigateway/latest/developerguide/integrating-api-with-aws-services-kinesis.html"
},
{
"text": "Kinesis Data Stream PutRecord API",
"link": "https://docs.aws.amazon.com/kinesis/latest/APIReference/API_PutRecord.html"
},
{
"text": "Using AWS Lambda with Amazon Kinesis",
"link": "https://docs.aws.amazon.com/lambda/latest/dg/with-kinesis.html"
}
]
},
"deploy": {
"text": [
"sam deploy --guided"
]
},
"testing": {
"text": [
"See the Github repo for detailed testing instructions."
]
},
"cleanup": {
"text": [
"Delete the stack: <code>sam delete</code>."
]
},
"authors": [
{
"name": "Thi Nguyen",
"image": "https://drive.google.com/file/d/188LpzUvUmHt1o7vzbwKw32S-fYabL-qY/view?usp=sharing",
"bio": "Solutions Architect @ AWS",
"linkedin": "https://www.linkedin.com/in/ndthi"
}
]
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading