CI Job Fails: Test_Ingester_ModeHandler & Data Race in Memberlist Test

[CharlieTLe]

Description:
The CI job failed during the execution of the tests. Specifically, two failures were encountered:

1. Test Failure:

   • Test: Test_Ingester_ModeHandler
   • Trace: The test Test_Ingester_ModeHandler failed with the following error messages:
     • instance localhost:0 in state PENDING
     • this instance owns no tokens

   These errors suggest that the test could not correctly handle the state or token ownership for the ingester.

2. Data Race Detected:

   • Test: TestRejoin in memberlist_client_test.go
   • Error: A data race was detected during the execution of the TestRejoin function in the KV memberlist implementation. This data race affects the dnsProviderMock.Resolve() and KV.discoverMembers() functions.

Logs:

• Full log available in the CI run: https://github.com/cortexproject/cortex/actions/runs/10984054033/attempts/1?pr=6226
• Relevant log snippets:
  • Error Trace: /__w/cortex/cortex/pkg/ingester/ingester_test.go:5321
  • Received unexpected error: instance localhost:0 in state PENDING
  • WARNING: DATA RACE
  • race detected during execution of test

Steps to Reproduce:

1. Run the CI job using make BUILD_IN_CONTAINER=false test.
2. Observe the failure during the execution of Test_Ingester_ModeHandler.
3. A data race is observed in the TestRejoin test.

Expected Behavior:

• The tests should pass without errors, and there should be no data race conditions during execution.

Actual Behavior:

• Tests failed with errors related to the ingester mode handler and data race in the memberlist client tests.

Environment:

• Docker container used in the build.
• Go version: 1.22.7

job-log-fail.txt

Priority: High