lambda policy size exceeds limit when used with multiple RestApi methods

[AmitBaranes]

short description:

Override Lambda Permission manually when creating a new API Gateway method.

Use Case

Recently I faced this issue when pointing multiple API Gateways methods to specific lambda :

The final policy size (XXX) is bigger than the limit (20480)

The feature allows users to bypass automatic permission creation (sort of "manual mode") to avoid this limitation.

I noticed that when using LambdaIntegration class the bind function adds permissions automatically to the lambda function based on the method URL.

My workaround was overriding the bind function with my own class ( see this post - https://stackoverflow.com/questions/59713522/cdk-override-bind-when-using-lambdaintegration?noredirect=1#comment105588249_59713522) and implement my own logic.

This feature can give more flexibility to the end-user and could give more control about the lambda permissions.

This is a 🚀 Feature Request

[nija-at]

@AmitBaranes -

I've re-purposed the issue title to define what the issue. While your suggestion to fix this does make it more flexible, it wouldn't be the right customer experience for a CDK user.

Looking to understand this a little more -

Could you provide the number of APIGateway methods that use the same lambda function? It would be useful if you can also get the generated policy from the CloudFormation template and put it on the issue.

Do you know which of these limits - https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_iam-limits.html - are you specifically encountering?

[SomayaB]

Closing this issue since there hasn't been a response in a while. Feel free to reopen.

[rehanvdm]

@nija-at "it wouldn't be the right customer experience for a CDK user." Why do you think this? Also it is the resource based policy which has limit of 2,048.

The problem is that LambdaIntegration calls the lambda function addPermission everytime for a new Method. The policy gets filled up quite quickly and then the stack fails stating that the policy size is too big. The solution is to create a variation of the LambdaIntegration class that does the binding but does not add a individual method policy for each API method but rather just give the whole API (any resource + method) access to the Lambda.

import lambda = require('@aws-cdk/aws-lambda');
import apigateway = require('@aws-cdk/aws-apigateway');

class LambdaIntegrationNoPermission extends apigateway.LambdaIntegration
{
    constructor(handler: lambda.IFunction, options?: apigateway.LambdaIntegrationOptions) {
        super(handler, options);
    }

    bind(method: apigateway.Method)
    {
        this['scope'] = method;
    }
}

const api = new apigateway.RestApi(this, id+"-api", {
            restApiName: id,
            deployOptions: { stageName: buildPros.Environment },
            defaultCorsPreflightOptions: {
                allowOrigins: apigateway.Cors.ALL_ORIGINS,
                allowMethods: apigateway.Cors.ALL_METHODS,
                allowHeaders: ["*"]
            },
            defaultIntegration: new LambdaIntegrationNoPermission(apiLambda, {proxy: true}),
        });

.... Add many methods and resources here ....

/* Manually add the permission, specifying with the API function arnForExecuteApi empty params means for all methods, paths and stages    */
apiLambda.addPermission(id + "ApiGWPermissions", {
           action: 'lambda:InvokeFunction',
           principal: new iam.ServicePrincipal('apigateway.amazonaws.com'),
           sourceArn: api.arnForExecuteApi()
       });

[andrew-terekhov]

@rehanvdm Is it possible to make the same trick with this['scope'] = method in python?

[AmitBaranes]

@andrew-terekhov I guess there is, you need to override the parent function. I'm not familiar with how to do that in python. but, I'm sure there is a way.

[andrew-terekhov]

@AmitBaranes Thank you for the response!
I've tried to do it in python but discovered that scope is a private attribute of AwsIntegration in TypeScript, so I can't access this attribute in python. I've tried to override bind method but I should call the parent method that adds permissions anyway.

[Iku-turso]

I'm getting burned by this as well :/

[rehanvdm]

@nija-at "it wouldn't be the right customer experience for a CDK user." Why do you think this? Also it is the resource based policy which has limit of 2,048.

The problem is that LambdaIntegration calls the lambda function addPermission everytime for a new Method. The policy gets filled up quite quickly and then the stack fails stating that the policy size is too big. The solution is to create a variation of the LambdaIntegration class that does the binding but does not add a individual method policy for each API method but rather just give the whole API (any resource + method) access to the Lambda.

import lambda = require('@aws-cdk/aws-lambda');
import apigateway = require('@aws-cdk/aws-apigateway');

class LambdaIntegrationNoPermission extends apigateway.LambdaIntegration
{
    constructor(handler: lambda.IFunction, options?: apigateway.LambdaIntegrationOptions) {
        super(handler, options);
    }

    bind(method: apigateway.Method)
    {
        this['scope'] = method;
    }
}

const api = new apigateway.RestApi(this, id+"-api", {
            restApiName: id,
            deployOptions: { stageName: buildPros.Environment },
            defaultCorsPreflightOptions: {
                allowOrigins: apigateway.Cors.ALL_ORIGINS,
                allowMethods: apigateway.Cors.ALL_METHODS,
                allowHeaders: ["*"]
            },
            defaultIntegration: new LambdaIntegrationNoPermission(apiLambda, {proxy: true}),
        });

.... Add many methods and resources here ....

/* Manually add the permission, specifying with the API function arnForExecuteApi empty params means for all methods, paths and stages    */
apiLambda.addPermission(id + "ApiGWPermissions", {
           action: 'lambda:InvokeFunction',
           principal: new iam.ServicePrincipal('apigateway.amazonaws.com'),
           sourceArn: api.arnForExecuteApi()
       });

Does not work anymore since the Method signature changed, refer to the newly opened ticket addressing this and the new solution here: #9327 (comment)

[sabornibhattacharya]

I have published a serverlessland pattern that can be used as a workaround meanwhile -

https://serverlessland.com/patterns/apigw-lambda-wildcard-resourcebasedpolicy-cdk

The solution replaces all other would-be created policy for each new integration with a wildcard resource-based policy.

[bbuechler]

I have published a serverlessland pattern that can be used as a workaround meanwhile -

https://serverlessland.com/patterns/apigw-lambda-wildcard-resourcebasedpolicy-cdk

The solution replaces all other would-be created policy for each new integration with a wildcard resource-based policy.

This worked smashingly for me, with one caveat. Since I had already hit the limit, CDK wouldn't deploy the fix since it would try to create the wildcard policy before removing the specific policies and that would cross the policy length threshold.

I had to first comment out a number of add_resource() & add_method() pairs prior to implementing the workaround, and redeploy my stack which removed the policy statements freeing up space. THEN I could implement the work around, uncomment the resources/methods, redeploy my stack, and verify there Lambda policy had ONLY the wildcard statement!

Thanks! 🥳