Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to typescript and response streaming #162

Merged
merged 37 commits into from
Nov 20, 2023
Merged

Switch to typescript and response streaming #162

merged 37 commits into from
Nov 20, 2023

Conversation

joehoyle
Copy link
Member

@joehoyle joehoyle commented Jul 27, 2023

This is a major version, with the following:

  • Switch the repo to use Typescript, which provides a lot more safety around the Lambda request args info, args etc.
  • Upgrade to Node v18
  • Use Lambda Response Streaming to we can return larger files and improve TTFB
  • Use newer API Gateway/Lambda Function URL request type to support Lambda function URLs
  • Remove support for CloudFormation
  • use the AWS SAM stack for local dev and testing
  • Added support for GIF resizing
  • New Readme / testing docs
  • Auto build the lambda file on publish release
  • Switch to Github Actions for testing

Todo:

  • Support for proxying signed requests
  • local server solution for local-server

@joehoyle joehoyle requested a review from rmccue July 28, 2023 14:01
@joehoyle joehoyle marked this pull request as ready for review July 28, 2023 14:02
package.json Outdated
"test": "docker run --rm -v `pwd`:/var/task -it --entrypoint='node' public.ecr.aws/lambda/nodejs:14 /var/task/test-filesize/index.js",
"update-test-fixtures": "docker run --rm -v `pwd`:/var/task -it --entrypoint='node' public.ecr.aws/lambda/nodejs:14 /var/task/test-filesize/index.js --update-fixtures",
"build-zip": "rm lambda.zip; zip -r --exclude='node_modules/aws-sdk/*' --exclude='node_modules/animated-gif-detector/test/*' lambda.zip ./node_modules/ index.js proxy-file.js lambda-handler.js",
"build": "sam build -u",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is sam actually added as a dependency? (Is it in aws-lambda?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it's part of https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html. I think we need to document it in CONTRIBUTING.md though

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK documented

src/lambda-handler.ts Outdated Show resolved Hide resolved
src/lambda-handler.ts Outdated Show resolved Hide resolved
src/lambda-handler.ts Outdated Show resolved Hide resolved
src/lib.ts Outdated Show resolved Hide resolved
src/lib.ts Outdated
}

// add invalid args
resolve({ data, info: { ...info, errors: errors.join(';') } });
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not return errors: string[]?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's just what it did already and I didn't want to change the interface

src/server.ts Outdated Show resolved Hide resolved
src/server.ts Outdated Show resolved Hide resolved
src/server.ts Outdated Show resolved Hide resolved
Handler: dist/lambda-handler.handler
Environment:
Variables:
S3_BUCKET: hmn-uploads
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this use a dummy name if it's just a template? Don't want to risk it accidentally being used

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is meant to be used to be able to do the integration test. There's no permissions to read / write the bucket it's just public

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Can we use the test bucket in that case? (Per compliance, not using prod data for testing/development.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is no different than having a test which does curl http://humanmade.com/robots.txt to check it's working, so I don't think that really falls under using prod data for testing / development. That's more to do with copying prod data to development, so it leaks data potentially to people who only have access to "test data".

@rmccue
Copy link
Member

rmccue commented Aug 8, 2023

Also, the coding style here is all over the place. Tachyon was already a bit not great on this, but would be a good time to clean that up

@joehoyle
Copy link
Member Author

joehoyle commented Aug 9, 2023

Ok just need to decide on coding standards. At the moment the whole project uses Prettier with tabs. Do we want to use hm eslint-config? It hasn't been updated in 3 years so I wasn't sure whether we actually wanted to use that or not.

@joehoyle
Copy link
Member Author

joehoyle commented Aug 9, 2023

@rmccue ok ready for another review.

@joehoyle
Copy link
Member Author

joehoyle commented Sep 4, 2023

Fixes #3

'X-Amz-Date',
'X-Amz-Security-Token',
] as const;
const presignedParams: { [K in ( typeof presignedParamNames )[number]]?: string } = {}; // eslint-disable-line no-unused-vars
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would probably be a bit cleaner if you split the type out I think, since it's relatively complex?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hoisting it out of the function I don't think really does much to reduce the complexity

src/lib.ts Outdated Show resolved Hide resolved
src/lib.ts Outdated Show resolved Hide resolved
src/lib.ts Show resolved Hide resolved
src/lib.ts Show resolved Hide resolved
@joehoyle joehoyle requested a review from rmccue September 21, 2023 07:22
@joehoyle joehoyle merged commit 8702dfa into master Nov 20, 2023
@joehoyle joehoyle deleted the v3-branch branch November 20, 2023 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants