workflows/check-by-name: Better error when base branch also fails

Previously, even if the check also failed on the base branch, it looked
like the PR introduced the failure.

We can easily have a better error message for such cases.

Meanwhile this also paves the road for something like
#256788

.github/workflows/check-by-name.yml

@@ -21,6 +21,16 @@ jobs:
         with:
           # pull_request_target checks out the base branch by default
           ref: refs/pull/${{ github.event.pull_request.number }}/merge
+          # Fetches the merge commit and its parents
+          fetch-depth: 2
+      - name: Pinning git hashes
+        run: |
+          echo "merged_sha=$(git rev-parse HEAD)" >> "$GITHUB_ENV"
+
+          # For pull_request_target this is the same as $GITHUB_SHA
+          echo "base_sha=$(git rev-parse HEAD^1)" >> "$GITHUB_ENV"
+
+          # We don't need it here, but the PR head would be HEAD^2
       - uses: cachix/install-nix-action@v23
       - name: Determining channel to use for dependencies
         run: |
@@ -51,4 +61,22 @@ jobs:
           # Passing --max-jobs 0 makes sure that we won't build anything
           nix-build "$nixpkgs" -A tests.nixpkgs-check-by-name --max-jobs 0
       - name: Running nixpkgs-check-by-name
-        run: result/bin/nixpkgs-check-by-name .
+        run: |
+          check() {
+            git checkout -q "$1"
+            result/bin/nixpkgs-check-by-name .
+          }
+
+          echo "Checking whether the check succeeds on the base branch $GITHUB_BASE_REF"
+          if check "$base_sha"; then
+            echo "The check succeeds on the base branch $GITHUB_BASE_REF, now checking the PR"
+            check "$merged_sha"
+          else
+            echo "The check fails on the base branch $GITHUB_BASE_REF, now checking the PR"
+            if check "$merged_sha"; then
+              echo "The check succeeds on the PR, so this fixes the base branch $GITHUB_BASE_REF, nicely done!"
+            else
+              echo "The check also fails on the PR, unknown if only this PRs changes would satisfy the check, the base branch $GITHUB_BASE_REF needs to be fixed"
+              false
+            fi
+          fi