No more test results in the pull request since 2.9.0

[adrianreber]

For our project we are using this GitHub Action (thanks a lot), but since 2.9.0 we no longer see the results posted in the corresponding pull request.

This was the last PR with 2.8.0 openhpc/ohpc#1848 and since then it switched to 2.9.0 and the pull request no longer contains the test information. (Actions log https://github.com/openhpc/ohpc/actions/runs/5646947939)

PR with 2.9.0 openhpc/ohpc#1860 (Actions log https://github.com/openhpc/ohpc/actions/runs/5890275118)

We are using following setup: https://github.com/openhpc/ohpc/blob/3.x/.github/workflows/test-analysis.yml

I am not sure if this a user error or if something changed. It seems to be related to 2.9.0.

Any suggestions what we might be doing wrong? Thanks.

[EnricoMi]

I think the issue here is that workflow Analyze Test Results runs after Validate workflow completed running on branch 3.x:

on:
  workflow_run:
    workflows: [Validate]
    types:
      - completed
    branches:
      - '3.x'

Your Validate workflow runs for push events on 3.x and pull_request events that target 3.x:

on:
  push:
    branches:
      - 3.x
  pull_request:
    branches:
      - 3.x

But pull_request events that target 3.x actually run on the branch of the respective PR, so they never trigger Validate for commits in a PR's branch, only push events do, when a PR is merged into 3.x.

I recommend you remove

    branches:
      - '3.x'

from the Analyze Test Results workflow.

[EnricoMi]

For instance, pull request #1872 from mghpcsim:3.x targeting 3.x triggered the Analyze Test Results workflow, as the pull_request event runs on branch 3.x (of the fork): https://github.com/openhpc/ohpc/actions/runs/6236154747 triggering https://github.com/openhpc/ohpc/actions/runs/6236354221.

Any PR with a branch other than 3.x does not trigger the Analyze Test Results workflow.

[adrianreber]

Thanks. That makes sense and of course was the problem.

The goal from the line was to only run the analysis if the pull request was opened against the 3.x branch and not if the remote branch is named 3.x.

If I remember it correctly the way publish-unit-test-results works for pull request from forked projects was that we need a job on the default branch that triggers when all tests have finished and the results have been uploaded.

We are running CI also on non default branches and did not want to have failures in the "Actions" tab because a PR against the non default branch was running.

Anyway, we are good. Thanks for taking a look at this.