-
Notifications
You must be signed in to change notification settings - Fork 928
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #2399 from dmahapatro/main
S3 notification event trigger Java-based AWS Lambda function
- Loading branch information
Showing
10 changed files
with
563 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,99 @@ | ||
# Resizing images uploaded to Amazon S3 with AWS Lambda | ||
|
||
The AWS SAM template deploys: | ||
- A Java-based Lambda function: | ||
- Powered by [Amazon Q Developer](https://aws.amazon.com/q/developer/) | ||
- [**NEW**] [Lambda SnapStart enabled for ARM64](https://aws.amazon.com/about-aws/whats-new/2024/07/aws-lambda-snapstart-java-functions-arm64-architecture/) based architecture | ||
- A source S3 bucket | ||
- A destination S3 bucket | ||
- The IAM resources required to run the application. | ||
|
||
The Lambda function consumes `ObjectCreated` events from an Amazon S3 bucket. | ||
The Lambda code checks the uploaded file is an image and creates a lower resolution version of the image in the destination bucket. | ||
|
||
Learn more about this pattern at Serverless Land Patterns: [https://serverlessland.com/patterns/s3-lambda](https://serverlessland.com/patterns/s3-lambda) | ||
|
||
Important: this application uses various AWS services, and there are costs associated with these services after the Free Tier usage. | ||
Please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. | ||
You are responsible for any AWS costs incurred. No warranty is implied in this example. | ||
|
||
## Requirements | ||
* [Java 21](https://docs.aws.amazon.com/corretto/latest/corretto-21-ug/downloads-list.html) | ||
* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have enough permissions to make necessary AWS service calls and manage AWS resources. | ||
* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured | ||
* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) | ||
* [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) (AWS SAM) installed | ||
|
||
## Deployment Instructions | ||
|
||
- Create a new directory, navigate to that directory in a terminal and clone the GitHub repository: | ||
``` | ||
git clone https://github.com/aws-samples/serverless-patterns | ||
``` | ||
- Change directory to the pattern directory: | ||
``` | ||
cd s3-lambda-resizing-java | ||
``` | ||
- From the command line, use AWS SAM to build and deploy the AWS resources for the pattern as specified in the `template.yml` file: | ||
``` | ||
sam build && sam deploy --guided | ||
``` | ||
- During the prompts: | ||
* Enter a stack name | ||
* Enter the desired AWS Region | ||
* Enter names for your source and destination S3 buckets. Make sure these are unique as S3 bucket names share a global namespace. | ||
* Allow SAM CLI to create IAM roles with the required permissions. | ||
|
||
Once you have run above once and saved arguments to a configuration file (`samconfig.toml`), | ||
you can use `sam deploy` in future to use these defaults. | ||
|
||
Note the outputs from the AWS SAM deployment process. These contain the resource names and/or ARNs which are used for testing. | ||
|
||
## How it works | ||
|
||
* Use the AWS CLI to upload an image to S3 | ||
* If the object is a .jpeg file, the code creates a 800x600 resolution image and saves it to the destination bucket. | ||
|
||
## Testing | ||
|
||
Run the following S3 CLI command to upload an image to the S3 bucket. | ||
Note, you must edit the {SourceBucketName} and {DestinationBucketName} placeholders with the name of the S3 Bucket which are provided in the stack outputs. | ||
|
||
```bash | ||
aws s3 cp './data/white_dog.jpeg' s3://{SourceBucketName} | ||
``` | ||
|
||
Run the following command to check that a new version of the image has been created in the destination bucket. | ||
|
||
```bash | ||
aws s3 ls s3://{DestinationBucketName} | ||
``` | ||
|
||
## Running JUnit 5 test | ||
To run the JUnit test `AppTest.java`, you need to make few changes: | ||
|
||
- Modify the [events/s3_event.json](./ResizerFunction/src/test/resources) file to reflect the source bucket name. Replace the value in the placeholder. | ||
- Replace the destination bucket name placeholder in the surefire plugin's environment variable placeholder in the `pom.xml` file. | ||
- Enable the test [`AppTest.java`](./ResizerFunction/src/test/java/resizer/AppTest.java) by removing the `@Disabled` annotation. | ||
- Change directory to `./ResizerFunction`. | ||
- Run `mvn clean test` | ||
|
||
You should see the test passing. | ||
The test uses [aws-lambda-java-tests](https://github.com/aws/aws-lambda-java-libs/tree/main/aws-lambda-java-tests) library. | ||
|
||
## Cleanup | ||
|
||
- Delete the objects created in the source and the destination bucket. | ||
- Delete the stack | ||
```bash | ||
sam delete --stack-name STACK_NAME | ||
``` | ||
- Confirm the stack has been deleted | ||
```bash | ||
aws cloudformation list-stacks --query "StackSummaries[?contains(StackName,'STACK_NAME')].StackStatus" | ||
``` | ||
---- | ||
|
||
Copyright 2024 Amazon.com, Inc. or its affiliates. All Rights Reserved. | ||
|
||
SPDX-License-Identifier: MIT-0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,85 @@ | ||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> | ||
<modelVersion>4.0.0</modelVersion> | ||
<groupId>org.example</groupId> | ||
<artifactId>resizer</artifactId> | ||
<version>1.0</version> | ||
<packaging>jar</packaging> | ||
<name>A sample app to resize images.</name> | ||
|
||
<properties> | ||
<maven.compiler.source>21</maven.compiler.source> | ||
<maven.compiler.target>21</maven.compiler.target> | ||
</properties> | ||
|
||
<dependencies> | ||
<dependency> | ||
<groupId>software.amazon.awssdk</groupId> | ||
<artifactId>s3</artifactId> | ||
<version>2.26.27</version> | ||
</dependency> | ||
<dependency> | ||
<groupId>com.amazonaws</groupId> | ||
<artifactId>aws-lambda-java-core</artifactId> | ||
<version>1.2.3</version> | ||
</dependency> | ||
<dependency> | ||
<groupId>com.amazonaws</groupId> | ||
<artifactId>aws-lambda-java-events</artifactId> | ||
<version>3.11.3</version> | ||
</dependency> | ||
<dependency> | ||
<groupId>com.amazonaws</groupId> | ||
<artifactId>aws-lambda-java-tests</artifactId> | ||
<version>1.1.1</version> | ||
</dependency> | ||
<dependency> | ||
<groupId>org.junit.jupiter</groupId> | ||
<artifactId>junit-jupiter-engine</artifactId> | ||
<version>5.10.2</version> | ||
<scope>test</scope> | ||
</dependency> | ||
<dependency> | ||
<groupId>org.junit.jupiter</groupId> | ||
<artifactId>junit-jupiter-api</artifactId> | ||
<version>5.10.2</version> | ||
<scope>test</scope> | ||
</dependency> | ||
<dependency> | ||
<groupId>org.slf4j</groupId> | ||
<artifactId>slf4j-simple</artifactId> | ||
<version>2.0.13</version> | ||
</dependency> | ||
</dependencies> | ||
|
||
<build> | ||
<plugins> | ||
<plugin> | ||
<groupId>org.apache.maven.plugins</groupId> | ||
<artifactId>maven-shade-plugin</artifactId> | ||
<version>3.5.0</version> | ||
<configuration> | ||
</configuration> | ||
<executions> | ||
<execution> | ||
<phase>package</phase> | ||
<goals> | ||
<goal>shade</goal> | ||
</goals> | ||
</execution> | ||
</executions> | ||
</plugin> | ||
<plugin> | ||
<groupId>org.apache.maven.plugins</groupId> | ||
<artifactId>maven-surefire-plugin</artifactId> | ||
<version>3.2.5</version> | ||
<configuration> | ||
<environmentVariables> | ||
<AWS_REGION>us-east-1</AWS_REGION> | ||
<DESTINATION_BUCKET_NAME>provide-destination-bucket-name-here</DESTINATION_BUCKET_NAME> | ||
</environmentVariables> | ||
</configuration> | ||
</plugin> | ||
</plugins> | ||
</build> | ||
</project> |
63 changes: 63 additions & 0 deletions
63
s3-lambda-resizing-java/ResizerFunction/src/main/java/resizer/App.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
package resizer; | ||
|
||
import com.amazonaws.services.lambda.runtime.Context; | ||
import com.amazonaws.services.lambda.runtime.RequestHandler; | ||
import com.amazonaws.services.lambda.runtime.events.S3Event; | ||
import com.amazonaws.services.lambda.runtime.events.models.s3.S3EventNotification.S3EventNotificationRecord; | ||
import software.amazon.awssdk.core.ResponseBytes; | ||
import software.amazon.awssdk.core.sync.RequestBody; | ||
import software.amazon.awssdk.regions.Region; | ||
import software.amazon.awssdk.services.s3.S3Client; | ||
import software.amazon.awssdk.services.s3.model.GetObjectRequest; | ||
import software.amazon.awssdk.services.s3.model.GetObjectResponse; | ||
import software.amazon.awssdk.services.s3.model.PutObjectRequest; | ||
|
||
import javax.imageio.ImageIO; | ||
import java.awt.*; | ||
import java.awt.image.BufferedImage; | ||
import java.io.ByteArrayOutputStream; | ||
|
||
public class App implements RequestHandler<S3Event, String> { | ||
private static final S3Client s3Client = S3Client.builder() | ||
.region(Region.of(System.getenv("AWS_REGION"))) | ||
.build(); | ||
private static final String DESTINATION_BUCKET = System.getenv("DESTINATION_BUCKET_NAME"); | ||
|
||
public String handleRequest(final S3Event s3event, final Context context) { | ||
try { | ||
S3EventNotificationRecord record = s3event.getRecords().getFirst(); | ||
String sourceBucket = record.getS3().getBucket().getName(); | ||
String sourceKey = record.getS3().getObject().getUrlDecodedKey(); | ||
|
||
// Get the image from S3 bucket | ||
GetObjectRequest getObjectRequest = GetObjectRequest.builder() | ||
.bucket(sourceBucket) | ||
.key(sourceKey) | ||
.build(); | ||
ResponseBytes<GetObjectResponse> getObjectResponseBytes = s3Client.getObjectAsBytes(getObjectRequest); | ||
BufferedImage originalImage = ImageIO.read(getObjectResponseBytes.asInputStream()); | ||
|
||
// Resize the image | ||
int newWidth = 800; | ||
int newHeight = 600; | ||
Image resizedImage = originalImage.getScaledInstance(newWidth, newHeight, Image.SCALE_SMOOTH); | ||
|
||
// Convert the resized image to a BufferedImage | ||
BufferedImage outputImage = new BufferedImage(newWidth, newHeight, BufferedImage.TYPE_INT_RGB); | ||
outputImage.getGraphics().drawImage(resizedImage, 0, 0, null); | ||
|
||
// Upload the resized image to the destination bucket | ||
ByteArrayOutputStream outputStream = new ByteArrayOutputStream(); | ||
ImageIO.write(outputImage, "jpg", outputStream); | ||
PutObjectRequest putObjectRequest = PutObjectRequest.builder() | ||
.bucket(DESTINATION_BUCKET) | ||
.key("resized_%s".formatted(sourceKey)) | ||
.build(); | ||
s3Client.putObject(putObjectRequest, RequestBody.fromBytes(outputStream.toByteArray())); | ||
|
||
return "Ok"; | ||
} catch (Exception e) { | ||
throw new RuntimeException(e); | ||
} | ||
} | ||
} |
59 changes: 59 additions & 0 deletions
59
s3-lambda-resizing-java/ResizerFunction/src/test/java/resizer/AppTest.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
package resizer; | ||
|
||
import com.amazonaws.services.lambda.runtime.events.S3Event; | ||
import com.amazonaws.services.lambda.runtime.tests.annotations.Event; | ||
import org.junit.jupiter.api.AfterEach; | ||
import org.junit.jupiter.api.Disabled; | ||
import org.junit.jupiter.params.ParameterizedTest; | ||
import software.amazon.awssdk.regions.Region; | ||
import software.amazon.awssdk.services.s3.S3Client; | ||
import software.amazon.awssdk.services.s3.model.ListObjectsRequest; | ||
import software.amazon.awssdk.services.s3.model.ListObjectsResponse; | ||
|
||
import static org.junit.jupiter.api.Assertions.assertEquals; | ||
import static org.junit.jupiter.api.Assertions.assertNotNull; | ||
|
||
@Disabled("Disabled until source and destination buckets are set up") | ||
public class AppTest { | ||
private static final S3Client s3Client = S3Client.builder() | ||
.region(Region.of(System.getenv("AWS_REGION"))) | ||
.build(); | ||
|
||
private static final String DESTINATION_BUCKET_NAME = System.getenv("DESTINATION_BUCKET_NAME"); | ||
|
||
@AfterEach | ||
public void cleanup() { | ||
ListObjectsRequest listObjectRequest = ListObjectsRequest.builder() | ||
.bucket(DESTINATION_BUCKET_NAME) | ||
.build(); | ||
ListObjectsResponse listObjectResponse = s3Client.listObjects(listObjectRequest); | ||
|
||
if (listObjectResponse.contents() != null) { | ||
listObjectResponse.contents().forEach(content -> | ||
s3Client.deleteObject(builder -> builder.bucket(DESTINATION_BUCKET_NAME).key(content.key())) | ||
); | ||
} | ||
} | ||
|
||
@ParameterizedTest | ||
@Event(value = "events/s3_event.json", type = S3Event.class) | ||
public void successfulResponse(S3Event input) { | ||
App app = new App(); | ||
String response = app.handleRequest(input, null); | ||
String sourceKey = input.getRecords().getFirst().getS3().getObject().getKey(); | ||
|
||
assertNotNull(input.getRecords().getFirst().getS3().getBucket().getName()); | ||
assertNotNull(sourceKey); | ||
|
||
assertEquals("Ok", response); | ||
|
||
ListObjectsRequest listObjectRequest = ListObjectsRequest.builder() | ||
.bucket(System.getenv("DESTINATION_BUCKET_NAME")) | ||
.build(); | ||
ListObjectsResponse listObjectResponse = s3Client.listObjects(listObjectRequest); | ||
|
||
assertNotNull(listObjectResponse); | ||
assertNotNull(listObjectResponse.contents()); | ||
assertEquals(1, listObjectResponse.contents().size()); | ||
} | ||
} |
38 changes: 38 additions & 0 deletions
38
s3-lambda-resizing-java/ResizerFunction/src/test/resources/events/s3_event.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
{ | ||
"Records": [ | ||
{ | ||
"eventVersion": "2.1", | ||
"eventSource": "aws:s3", | ||
"awsRegion": "us-east-1", | ||
"eventTime": "2024-07-03T19:37:27.192Z", | ||
"eventName": "ObjectCreated:Put", | ||
"userIdentity": { | ||
"principalId": "AWS:AIDAINPONIXQXHT3IKHL2" | ||
}, | ||
"requestParameters": { | ||
"sourceIPAddress": "205.255.255.255" | ||
}, | ||
"responseElements": { | ||
"x-amz-request-id": "D82B88E5F771F645", | ||
"x-amz-id-2": "vlR7PnpV2Ce81l0PRw6jlUpck7Jo5ZsQjryTjKlc5aLWGVHPZLj5NeC6qMa0emYBDXOo6QBU0Wo=" | ||
}, | ||
"s3": { | ||
"s3SchemaVersion": "1.0", | ||
"configurationId": "828aa6fc-f7b5-4305-8584-487c791949c1", | ||
"bucket": { | ||
"name": "<provide-source-bucket-name-here>", | ||
"ownerIdentity": { | ||
"principalId": "A3I5XTEXAMAI3E" | ||
}, | ||
"arn": "arn:aws:s3:::lambda-artifacts-deafc19498e3f2df" | ||
}, | ||
"object": { | ||
"key": "white_dog.jpeg", | ||
"size": 1305107, | ||
"eTag": "b21b84d653bb07b05b1e6b33684dc11b", | ||
"sequencer": "0C0F6F405D6ED209E1" | ||
} | ||
} | ||
} | ||
] | ||
} |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.