Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deployment Fails Due to Lambda MemorySize Exceeding 3008 MB #48

Open
erosmontin opened this issue Dec 7, 2024 · 8 comments
Open

Deployment Fails Due to Lambda MemorySize Exceeding 3008 MB #48

erosmontin opened this issue Dec 7, 2024 · 8 comments

Comments

@erosmontin
Copy link

I encountered an issue while deploying versions 2.9.3 and 2.9.1 of the AWS Media Replay Engine. The deployment failed in both us-east-1 and us-east-2 regions, regardless of whether I deployed from my laptop or an EC2 instance while following the instructions in the repository.

The deployment gets stuck due to the following error:
Resource handler returned message: "'MemorySize' value failed to satisfy constraint: Member must have a value less than or equal to 3008 (Service: Lambda, Status Code: 400, Request ID: xxx"

@jjbainAWS
Copy link

Hi, thanks for raising this issue, I am currently trying to reproduce this error. I just followed the instructions provided in the repository and wasn't able to reproduce this error.

Could you please provide the commands you used in the EC2 instance to try to deploy MRE?

@erosmontin
Copy link
Author


Environment Setup

  1. Created an EC2 instance:

    • Amazon Machine Image (AMI): Amazon Linux 2023 (AL2023).
    • Instance Type: t2.large (minimum requirement).
    • Architecture: x86_64.
    • Storage: Attached a root volume of 16 GB (gp3).
  2. Created an IAM policy:

    • Updated env-role-permissions.json:
      • Replaced ${AWS_ACCOUNT_ID} with my actual AWS account ID.
      • Replaced ${AWS_REGION} with us-east-1.
  3. Attached the policy to a Role and associated to the EC2 machine

  4. Cloned the MRE repository.


Commands Executed

  1. Navigated to the /deployment/ directory.

  2. Ran the following commands:

    #!/bin/bash
    chmod +x env-init.sh
    ./env-init.sh
    • Encountered permission issues, so I used:
      sudo ./env-init.sh
  3. Set up region and version:

    REGION=us-east-1
    VERSION=2.9.1
  4. Ran the deployment script:

    ./build-and-deploy.sh --enable-ssm-high-throughput --version $VERSION --region $REGION --admin-email [email protected]

Error Encountered

During deployment, the following error occurred for a Lambda function resource:

This AWS::Lambda::Function resource is in a CREATE_FAILED state.

Resource handler returned message: "'MemorySize' value failed to satisfy constraint: Member must have value less than or equal to 3008 (Service: Lambda, Status Code: 400, Request ID: 65d3a954-58ed-4f5f-b595-a6cc9c142b43)" (RequestToken: d55c298f-10b1-6d14-4240-f12ac6f415de, HandlerErrorCode: InvalidRequest)

Let me know if additional details are needed or if specific debugging steps are recommended.

@jjbainAWS
Copy link

I see thanks for the information, this is helpful.
What I am immediately noticing is that you ran ./env-init.sh in the EC2 instance, this script is meant to be copied and run as EC2 user data, not in the EC2 instance after instantiation. You can copy the code and paste it into the user data field when configuring your EC2 instance.

Additionally, you will need to create a virtual environment by running the command

source mredeploy-env/bin/activate

after the env-init.sh completes.

Once the virtual environment is started, you can deploy MRE.

@erosmontin
Copy link
Author

thank you so much for your fast answer, I'll try this approach during the weekend.
Thanks a lot!

@jjbainAWS
Copy link

Yes no problem, let us know how it goes.

@imcarp
Copy link
Contributor

imcarp commented Dec 13, 2024

@erosmontin - want to bring attention to: https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html

"New AWS accounts have reduced concurrency and memory quotas. AWS raises these quotas automatically based on your usage."

So this could also be related as well. Keep us posted!

@erosmontin
Copy link
Author

hi folks, thank you so much for your help, I am still facing the same issue:

aws-mre-segment-caching | 3/7 | 7:25:02 PM | CREATE_IN_PROGRESS | AWS::Lambda::Function | Mre-SegmentCaching (MreSegmentCaching4BB22A99) Resource creation Initiated
aws-mre-segment-caching | 3/7 | 7:25:02 PM | CREATE_FAILED | AWS::Lambda::Function | Mre-SegmentCaching (MreSegmentCaching4BB22A99) Resource handler returned message: "'MemorySize' value failed to satisfy constraint: Member must have value less than or equal to 3008 (Service: Lambda, Status Code: 400, Request ID: b67b62d7-76e4-4e5d-9d8f-76bc0d7988ba)" (RequestToken: 779a2c9e-8a3e-8fe7-d87b-c025a10aa55f, HandlerErrorCode: InvalidRequest)
aws-mre-segment-caching | 3/7 | 7:25:02 PM | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack | aws-mre-segment-caching The following resource(s) failed to create: [MreSegmentCaching4BB22A99]. Rollback requested by user.

Do you have any suggestions for how to move forward? Could a different version work?

@imcarp
Copy link
Contributor

imcarp commented Dec 16, 2024

Could you try to create a Lambda function from scratch via the console (same region as where you were deploying MRE), set memory higher than 3GB, and see if you get an error? Trying to determine if we're dealing with account quota/limits or a problem with the deployment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants