Jk/cumulus 3940 to main from #3877 (#3877) #3890
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Update message recovery/granule write logic to properly use esClient This commit updates the following: - esClient is properly passed through api lambda/lib methods such that write granules calls from process-s3-dead-letter-archive can pass in an instance of EsClient rather than relying on default per-granule object/client behavior - The API endpoint and related code are updated such that maxDbPool, concurrency and batchSize are exposed as endpoint options, allowing user customization of tuning behavior for the DLA recovery tool - Minor typing/call fixes * Update Core to allow DLA recovery configuration This commit updates: - archive/cumulus/example to pass through memory configuration options to the fargate task definition * Add api performance test * Update docs/changelog * Update CHANGELOG and documentation * Update CHANGELOG * Fix linting * Fix units * Update dead letter archive feature doc * Update test spec * Update logging, make perf test script executable * Fix broken package.json ava exclusion configuration * Add zod parsing to dead letter endpoint * Update tf-modules/archive/async_operation.tf Co-authored-by: jennyhliu <[email protected]> * Update tf-modules/archive/async_operation.tf Co-authored-by: jennyhliu <[email protected]> * Address db pool configuration concern in PR * Update env config passthroughs/make log/docs consistent * Update tf-modules/archive/async_operation.tf Co-authored-by: jennyhliu <[email protected]> * Update tf-modules/archive/async_operation.tf Co-authored-by: jennyhliu <[email protected]> * Update per PR suggestion * Update concurrency defaults for consistency * Update startAsyncOperations to allow for optional container names * Update dead letter archive endpoint to specify new container name * Update API defaults/units to 30 to match system defaults * Fix defaults for endpoint tests * Add changed params to demonstrate payload handling * Updarte coverage metric Updated code in this module doesn't significantly impact test coverage, other than increasing the denominator. * fixup * Update performance tests to match documented defaults --------- Co-authored-by: jennyhliu <[email protected]>
Nnaga1
reviewed
Jan 2, 2025
Jkovarik
changed the title
Nnaga1
approved these changes
Jan 3, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary: Summary of changes
Addresses CUMULUS-3940
Changes
This PR brings forward the merged code from #3877 on the 18.5.x release series.
PR Checklist