Label removal from wrapped object does not trigger label removal on managed cluster

[andreyod]

Describe the bug
When label is removed from an object wrapped by ManifestWork, it is not removed from the object in the managed cluster.

To Reproduce

1. Setup OCM with https://open-cluster-management.io/getting-started/quick-start/
2. Use kubectl to create a ManifestWork on the hub with the following yaml file. Note the test: test label in the deployment metadata.

apiVersion: work.open-cluster-management.io/v1
kind: ManifestWork
metadata:
  namespace: cluster1
  name: my-first-work
spec:
  workload:
    manifests:
      - apiVersion: v1
        kind: ServiceAccount
        metadata:
          namespace: default
          name: my-sa
      - apiVersion: apps/v1
        kind: Deployment
        metadata:
          namespace: default
          name: nginx-deployment
          labels:
            app: nginx
            test: test
        spec:
          replicas: 1
          selector:
            matchLabels:
              app: nginx
          template:
            metadata:
              labels:
                app: nginx
            spec:
              serviceAccountName: my-sa
              containers:
                - name: nginx
                  image: nginx:1.14.2
                  ports:
                    - containerPort: 80

3. Ensure deployment nginx-deployment in cluster1 is labeled with the test label
   kubectl --context kind-cluster1 get deployment nginx-deployment -o yaml
4. Edit the ManifestWork and remove the test label.
   kubectl --context kind-hub edit manifestworks my-first-work -n cluster1
5. Deployment nginx-deployment in cluster1 is still labeled with the test label.

Expected behavior
In step 6 above, I expect the test label to be removed.

Environment ie: OCM version, Kubernetes version and provider:
kind v0.19.0 go1.19 linux/amd64

Additional context
Add new label and update a label value works as expected.

[qiujian16]

we use openshift/library-go to apply resources, the label is always merged. see mergemap here https://github.com/openshift/library-go/blob/master/pkg/operator/resource/resourcemerge/object_merger.go#L141

To remove the label, you will need to explicitly set it by adding a "-" suffix, e.g.

labels:
  test: test-

[andreyod]

From looking at https://github.com/openshift/library-go/blob/master/pkg/operator/resource/resourcemerge/object_merger.go#L141 , I can see that the key is checked for "-" suffix, not the value.
Anyway this edit is invalid for labels. Valid label key and value must begin and end with an alphanumeric character ([a-z0-9A-Z]) according to https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set

[qiujian16]

/assign

[github-actions]

This issue is stale because it has been open for 120 days with no activity. After 14 days of inactivity, it will be closed. Remove the stable label to prevent this issue from being closed.