🐛 fix race condition between work status update and deletion. #74
Conversation
morvencao
force-pushed
the
br_fix_deletion
branch
from
c08a864
to
dbacf24
Compare
|
/assign @qiujian16 @skeeey |
| return nil, errors.NewInternalError(err) | ||
| } | ||
| // ensure the resource version of the work is not outdated | ||
| if newResourceVersion < lastResourceVersion { |
There was a problem hiding this comment.
will this be a problem we return error here? since we publish but do not update local store?
There was a problem hiding this comment.
If we only consider the deletion case, there might not be an issue because the resource will be deleted later.
There was a problem hiding this comment.
this may not be a problem, if the resource version does not match, that means
- the resource spec has beed updated on the source
- the resource spec has beed updated on the agent local cache
because we publish the status update with resource version, so even if the update request is sent, the source should not accept this request.
for agent, this update is not for the current newest resource, so we should also reject it
There was a problem hiding this comment.
added some comments in 3ef6f5d for this.
morvencao
force-pushed
the
br_fix_deletion
branch
2 times, most recently
from
33dfdba
to
6a28efa
Compare
Signed-off-by: morvencao <[email protected]>
morvencao
force-pushed
the
br_fix_deletion
branch
from
6a28efa
to
05750c6
Compare
Signed-off-by: morvencao <[email protected]>
|
@qiujian16 Another look? |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: morvencao, qiujian16 The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-merge-bot
bot
merged commit 7bd852f
into
open-cluster-management-io:main
Summary
This PR mainly addresses a race condition where a work status update can overwrite the DeletionTimestamp, preventing the resource from being deleted. The issue occurs when an update event is followed immediately by a delete event from the source, causing the agent to skip deletion.
To resolve this, we now verify the resource version before updating the agent store for work status update.
Additionally, any update event that follows a delete event for the same resource from the source will be ignored from agent side.
In this PR we've also removed a duplicate debug log that printed the received event. (that is printed in baseclient.go)
Related issue(s)
Fixes #