We have enabled retry on failure mechanism in test plans for our CI. Whenever there is a flaky test that requires more than one iteration to succeed, the output is an unstable build on the CI (we are using Jenkins). This is the command we run:

xcodebuild test ... | xcbeautify --report junit --preserve-unbeautified --report-path some_path

The junit.xml output file from xcbeautify doesn't filter out failures if there is a retry.

I don't know the codebase but at a glance I guess that we could add some retry regex/logic here to filter out previous attempts if there was a successful retry.

I'm happy to contribute to the project if you think this is a valid issue. Any hint would also be appreciated. Wdyt?