Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement matching API to allow updating task list partition config #6472

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

Shaddoll
Copy link
Contributor

@Shaddoll Shaddoll commented Nov 2, 2024

Detailed Description
Define and implement new grpc APIs for matching:

  • UpdateTaskListPartitionConfig: This API is used by frontend service to forward update request initiated mostly from CLI tool to update the partition config of a task list and notify all partitions of the task list to update their cache of task list partition config.
  • RefreshTaskListPartitionConfig: This API is mainly used by matching service to notify a task list partition to update its cache of task list partition config. It can also be used by frontend service to forward request initiated from admin CLI tool to sync the cache of task list partition config if something goes wrong.

Impact Analysis

  • Backward Compatibility: Yes.
  • Forward Compatibility: Yes.

Testing Plan

  • Unit Tests: Yes.
  • Persistence Tests: No.
  • Integration Tests: Will be added in a separate PR.
  • Compatibility Tests: No.

Rollout Plan

  • What is the rollout plan?
  • Does the order of deployment matter? No.
  • Is it safe to rollback? Does the order of rollback matter? It's safe to rollback and the order doesn't matter.
  • Is there a kill switch to mitigate the impact immediately? We don't need a kill switch.

This PR introduces new matching API allowing us to Update and Refresh task list partition config which is stored in database. We'll have another PR exposing these endpoints in frontend, create CLI command for these endpoints and have integration tests covering this.

Copy link

codecov bot commented Nov 4, 2024

Codecov Report

Attention: Patch coverage is 95.28796% with 18 lines in your changes missing coverage. Please review.

Project coverage is 80.22%. Comparing base (a926dd7) to head (8eab0eb).
Report is 4 commits behind head on master.

Files with missing lines Patch % Lines
service/matching/tasklist/task_list_manager.go 89.91% 9 Missing and 3 partials ⚠️
service/matching/handler/engine.go 91.30% 4 Missing and 2 partials ⚠️
Additional details and impacted files
Files with missing lines Coverage Δ
client/matching/client.go 90.74% <100.00%> (+0.74%) ⬆️
common/types/mapper/proto/matching.go 96.59% <100.00%> (+0.43%) ⬆️
common/types/matching.go 100.00% <100.00%> (ø)
service/matching/config/config.go 100.00% <100.00%> (ø)
service/matching/handler/handler.go 100.00% <100.00%> (ø)
service/matching/tasklist/db.go 86.11% <100.00%> (+2.24%) ⬆️
service/matching/tasklist/identifier.go 100.00% <100.00%> (ø)
service/matching/tasklist/matcher.go 100.00% <100.00%> (ø)
service/matching/handler/engine.go 78.51% <91.30%> (+0.92%) ⬆️
service/matching/tasklist/task_list_manager.go 74.20% <89.91%> (+2.11%) ⬆️

... and 15 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a926dd7...8eab0eb. Read the comment docs.

@Shaddoll Shaddoll changed the title WIP: UpdateTaskListPartitionConfig Implement matching API to allow updating task list partition config Nov 5, 2024
@Shaddoll Shaddoll force-pushed the update branch 4 times, most recently from 3bc2459 to 8cd0c7c Compare November 5, 2024 20:19
domainID := request.DomainUUID
taskListName := request.TaskList.GetName()
taskListKind := request.TaskList.GetKind()
taskListType := persistence.TaskListTypeDecision
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: this is a little confusing to read to default like this, I'd suggest setting it to an zero value and switch below rather than defaulting

defer sw.Stop()

if ok := h.userRateLimiter.Allow(quotas.Info{Domain: domainName}); !ok {
return nil, hCtx.handleErr(errMatchingHostThrottle)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: a rate-limit event seems unlikely and probably worthy of a log

if err != nil {
var e *types.EntityNotExistsError
if !errors.As(err, &e) {
c.Stop()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

under what conditions would this occur? Could you add a comment if there's forseeable ones?

c.partitionConfigLock.RUnlock()
return
c.logger.Debug("get task list partition config from db", tag.Dynamic("root-partition", c.taskListID.GetRoot()), tag.Dynamic("config", config))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the log key config may conflict with other log structs using the same key in search indices and will result in the entire log-line being dropped. I'd just make it more unique with a name like tasklist-partition-config to avoid this.

Copy link
Contributor

@davidporter-id-au davidporter-id-au left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some minor comments, but broadly I can't fault anything

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants