New serverless pattern - API GW to SQS to Lambda to DDB #1480
Conversation
|
This is missing the example-pattern.json metadata file. Please add so the pattern can be tested. |
|
Missing example-pattern.json file is added now. |
|
@mavi888 Please let me know for any questions or outstanding things to be addressed before this can be merged. |
|
|
||
| This pattern explains how to deploy a SAM application with Amazon API Gateway, Amazon SQS, AWS Lambda, and Amazon DynamoDB. When an HTTP POST request is made to the Amazon API Gateway endpoint, Gateway authorizes the request by checking Basic auth credentials and on valid credentials, request payload is sent to Amazon Simple Queue Service. AWS Lambda function consumes event from the Queue and inserts the event/payload into the Amazon DynamoDB table. Amazon Simple Queue Service is also configured with a Dead Letter Queue where events are sent when retries to process those messages are repeatedly failed. | ||
|
|
||
| Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example. |
There was a problem hiding this comment.
this is a great pattern and very useful, so i think it would be good to mention in the intro of this pattern why is useful.
There was a problem hiding this comment.
Expanded the intro section along with some key benefits
apigw-sqs-lambda-ddb/README.md
Outdated
| This pattern was contributed by Ravi Kiran Ganji. | ||
|
|
||
| ---- | ||
| Copyright 2021 Amazon.com, Inc. or its affiliates. All Rights Reserved. |
apigw-sqs-lambda-ddb/README.md
Outdated
|
|
||
| 1. Delete the stack | ||
| ```bash | ||
| aws cloudformation delete-stack --stack-name STACK_NAME |
apigw-sqs-lambda-ddb/README.md
Outdated
| Once the application is deployed: | ||
| - Retrieve the HttpApiEndpoint value from CloudFormation Outputs | ||
| - Retrieve the username and password from Secrets Manager in AWS Console. | ||
| - Invoke the endpoint from Postman using some json payload and verify the payload saved in DynamoDB. |
There was a problem hiding this comment.
can you make the command with curl, so its easier to copypaste in the cli
apigw-sqs-lambda-ddb/README.md
Outdated
|
|
||
| Once the application is deployed: | ||
| - Retrieve the HttpApiEndpoint value from CloudFormation Outputs | ||
| - Retrieve the username and password from Secrets Manager in AWS Console. |
There was a problem hiding this comment.
Secret manager should be mentioned before as it was not presented earlier.
There was a problem hiding this comment.
Secrets manager is added to intro section
apigw-sqs-lambda-ddb/README.md
Outdated
| Example POST Request: https://{HttpApiEndpoint}.execute-api.us-east-1.amazonaws.com/submit | ||
| - Request Header: "Content-Type: application/json" | ||
| - Request Header: "Authorization: Basic <credentials>" (where credentials is the Base64 encoding of ID and password joined by a single colon :) | ||
| - Request Body: {"eventId":"value1", "message":"event message for testing"} |
There was a problem hiding this comment.
what should we see after we run this? What kind of information people testing this should look for?
There was a problem hiding this comment.
Expanded the whole testing section to add more details
apigw-sqs-lambda-ddb/README.md
Outdated
| aws cloudformation list-stacks --query "StackSummaries[?contains(StackName,'STACK_NAME')].StackStatus" | ||
| ``` | ||
|
|
||
| This pattern was contributed by Ravi Kiran Ganji. |
There was a problem hiding this comment.
this can be removed as we do the attribution differently
| - Messages from SQS is posted to a lambda function to process them. | ||
| - Lambda function receives the messages from SQS and saves them into a DynamoDB table. | ||
|
|
||
| ## Testing |
There was a problem hiding this comment.
as you add the DLQ it would be great to have an example on how to test that that is working.
There was a problem hiding this comment.
Added a section to test DLQ
|
This is a great pattern and very useful, that is why i would love to see a better readme. Please address the comments and I will merge this PR |
|
@mavi888 Appreciate the feedback. I have updated the readme to address your feedback. Please take a look and let me know for any further comments. |
|
Thanks for submitting this pattern. |
Description of changes: This pattern explains how to deploy a SAM application with Amazon API Gateway, Amazon SQS, AWS Lambda, and Amazon DynamoDB. When an HTTP POST request is made to the Amazon API Gateway endpoint, Gateway authorizes the request by checking Basic auth credentials and on valid credentials, request payload is sent to Amazon Simple Queue Service. AWS Lambda function consumes event from the Queue and inserts the event/payload into the Amazon DynamoDB table. Amazon Simple Queue Service is also configured with a Dead Letter Queue where events are sent when retries to process those messages are repeatedly failed. Diagram and more details are also documented in README file.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.