Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parameter to power-tune only cold starts #206

Merged
merged 40 commits into from
Sep 16, 2024

Conversation

mriccia
Copy link
Contributor

@mriccia mriccia commented May 24, 2023

Implement #176

This is another implementation based on #177

With this implementation the mechanism for publishing new versions is decoupled, and the Step Function workflow implements a loop to publish the versions sequentially.
This is done to avoid hitting the Timeout on the Lambda function publishing the Versions.

Copy link
Owner

@alexcasalboni alexcasalboni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finally reviewed this! Thank you again for the amazing contribution 🙏

I have quite a few comments & questions, and I still need to merge the latest master branch into this.

lambda/initializer.js Outdated Show resolved Hide resolved
lambda/publisher.js Outdated Show resolved Hide resolved
lambda/publisher.js Outdated Show resolved Hide resolved
lambda/publisher.js Outdated Show resolved Hide resolved
lambda/publisher.js Outdated Show resolved Hide resolved
lambda/utils.js Outdated Show resolved Hide resolved
template.yml Outdated Show resolved Hide resolved
template.yml Outdated Show resolved Hide resolved
@@ -5,7 +5,8 @@
"Effect": "Allow",
"Action": [
"lambda:InvokeFunction",
"lambda:GetFunctionConfiguration"
"lambda:GetFunctionConfiguration",
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sfloresk all the Terraform edits here will need to be ported to the new repo :)

Copy link
Contributor Author

@mriccia mriccia May 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test/unit/test-lambda.js Show resolved Hide resolved
@alexcasalboni
Copy link
Owner

Ok, merging the latest wasn't too bad after all :)

Copy link
Contributor Author

@mriccia mriccia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional comments: need to update all docs/testing instructions

@@ -5,7 +5,8 @@
"Effect": "Allow",
"Action": [
"lambda:InvokeFunction",
"lambda:GetFunctionConfiguration"
"lambda:GetFunctionConfiguration",
Copy link
Contributor Author

@mriccia mriccia May 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mriccia mriccia marked this pull request as draft May 8, 2024 16:56
@mriccia mriccia marked this pull request as ready for review May 16, 2024 11:04
@mriccia
Copy link
Contributor Author

mriccia commented May 16, 2024

Implemented the Terraform changes in the separate repo, happy to open a PR once we have the green light on the changes here
sfloresk/terraform-aws-lambda-power-tuning@main...mriccia:terraform-aws-lambda-power-tuning:main

Repository owner deleted a comment from mriccia May 20, 2024
Copy link
Owner

@alexcasalboni alexcasalboni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, almost there 🚀 🚀 🚀

test/unit/test-lambda.js Show resolved Hide resolved
lambda/cleaner.js Outdated Show resolved Hide resolved
lambda/utils.js Show resolved Hide resolved
// use results (which include logs) to compute average duration ...

const durations = utils.parseLogAndExtractDurations(results);
if (onlyColdStarts) {
// we care about the init duration in this case
const initDurations = utils.parseLogAndExtractInitDurations(results);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mriccia this is new - I am manually adding the init duration to the billed duration. This might not be the ideal solution for all cases, but it does work for what we need when onlyColdStarts=true. Does that make sense to you?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's a problem with this. The cost is derived from the durations array, now that we're adding the initDuration in case of cold start the cost calculation will be inaccurate.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the way the duration and the cost are calculated.
Please take a look when you have a moment, there are some Tests missing (planning to add early next week), but should be good to go if you're happy with this.

// defaulting the index to 0 as the index is required for onlyColdStarts
let aliasToInvoke = utils.buildAliasString(lambdaAlias, onlyColdStarts, 0);
// We need the architecture, regardless of onlyColdStarts or not
const {architecture, isPending} = await utils.getLambdaConfig(lambdaARN, aliasToInvoke);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at what the Initializer is doing (very similarly), I would consider merging utils.getLambdaPower and utils.getLambdaConfig since they are both using GetFunctionConfigurationCommand and they're simply retrieving different fields from the result.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not blocking for this PR - just a reminder for myself in the future :)

template.yml Outdated Show resolved Hide resolved
lambda/utils.js Show resolved Hide resolved
return {
power: config.MemorySize,
// we need to fetch env vars only to add a new one and force a cold start
description: config.Description,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned in another comment, this implementation of getLambdaPower has become very similar to getLambdaConfig below. It used to make more sense because it was a specialized version that only cared about config.MemorySize and returned a single value. But now it returns a structure with various config parameters, so I'd consider merging getLambdaPower and getLambdaConfig back into one single implementation where we fetch all the config parameters we need.

lambda/utils.js Show resolved Hide resolved
template.yml Show resolved Hide resolved
@alexcasalboni alexcasalboni merged commit cea8f07 into alexcasalboni:master Sep 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants