Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test reporting on GitHub #4

Open
wants to merge 30 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
24427c4
Publish test results for unit tests
robin-aws Oct 29, 2021
8ea84fe
Never say never(), apparently :)
robin-aws Oct 29, 2021
25a42b6
Hopefully fix yaml error
robin-aws Oct 29, 2021
f37e6ee
Quote the wildcard instead
robin-aws Oct 29, 2021
3a93a88
Potential issue with dots in file names
robin-aws Oct 29, 2021
05715c3
Ah that's simpler
robin-aws Oct 29, 2021
b7f7d3c
Try a different reporter option
robin-aws Oct 29, 2021
8bab3f1
Need slashes for regular expression on artifact names
robin-aws Oct 29, 2021
a8db2e3
Fix regular expression
robin-aws Oct 29, 2021
a47d76d
Let's try without the regexp
robin-aws Oct 29, 2021
448ce18
Whoops, eliminated the wrong dimension
robin-aws Oct 29, 2021
6c51764
Are spaces the problem?
robin-aws Oct 29, 2021
5860509
Try not using artifacts for now
robin-aws Oct 29, 2021
5fdfb6a
Try TRX
robin-aws Oct 30, 2021
2acce77
Try to work around action not finding artifacts directly
robin-aws Oct 30, 2021
924e8de
Uploading artifacts might help ya dingus
robin-aws Oct 30, 2021
b2e89ad
Need to check out apparently
robin-aws Oct 30, 2021
a4f9d80
Fix upload paths
robin-aws Oct 30, 2021
0a8313b
Single path
robin-aws Oct 30, 2021
3711d0c
Dammit
robin-aws Oct 30, 2021
5170ad8
Fix ordering
robin-aws Oct 30, 2021
f58a71c
Give artifacts parameter another try
robin-aws Oct 31, 2021
fe5f9f6
Add report for integration tests too
robin-aws Oct 31, 2021
3d5e297
Oh yeah, right
robin-aws Oct 31, 2021
ddddb52
Try breaking a test
robin-aws Oct 31, 2021
b0dd800
Always upload artifacts
robin-aws Oct 31, 2021
e6462bf
Separate test report job, to support PRs from forks
robin-aws Nov 5, 2021
ace9e8c
Revert "Try breaking a test"
robin-aws Nov 5, 2021
2a45566
Remove unused logger
robin-aws Nov 5, 2021
7e9447e
Merge branch 'master' into test-reporting-on-github
robin-aws Nov 8, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions .github/workflows/msbuild.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,6 @@ jobs:

build:
runs-on: ${{ matrix.os }}

strategy:
matrix:
## Windows jobs fail on some lit tests and sometimes fail even to
Expand Down Expand Up @@ -126,4 +125,9 @@ jobs:
unzip dafny/Package/CI.zip -d unzippedRelease
- name: Run integration tests
run: |
XUNIT_SHARD=${{ matrix.shard }} XUNIT_SHARD_COUNT=5 DAFNY_RELEASE=$PWD/unzippedRelease/dafny dotnet test -v:n dafny/Source/IntegrationTests/IntegrationTests.csproj
XUNIT_SHARD=${{ matrix.shard }} XUNIT_SHARD_COUNT=5 DAFNY_RELEASE=$PWD/unzippedRelease/dafny dotnet test -v:n --logger trx dafny/Source/IntegrationTests/IntegrationTests.csproj
- uses: actions/upload-artifact@v2
if: always()
with:
name: integration-test-results-${{ matrix.os }}
path: dafny/Source/IntegrationTests/TestResults/*.trx
29 changes: 29 additions & 0 deletions .github/workflows/test-report.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: 'Test Report'
on:
workflow_run:
workflows: ['Run XUnit tests', 'Build and Test']
types:
- completed
jobs:
report:
name: "Download and Publish Test Results"
needs: build
runs-on: ubuntu-latest
if: always()

steps:
- uses: actions/checkout@v2
with:
submodules: recursive

- name: Download Artifacts
uses: actions/download-artifact@v2
with:
path: artifacts

- name: Publish Test Results
uses: dorny/test-reporter@v1
with:
name: Test Results
path: 'artifacts/**/*.trx'
reporter: dotnet-trx
11 changes: 8 additions & 3 deletions .github/workflows/xunit-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,13 @@ jobs:
- name: Build
run: dotnet build --no-restore ${{env.solutionPath}}
- name: Run DafnyLanguageServer Tests
run: dotnet test --no-restore --verbosity normal Source/DafnyLanguageServer.Test
run: dotnet test --no-restore --verbosity normal --logger trx Source/DafnyLanguageServer.Test
- name: Run DafnyPipeline Tests
run: dotnet test --no-restore --verbosity normal Source/DafnyPipeline.Test
run: dotnet test --no-restore --verbosity normal --logger trx Source/DafnyPipeline.Test
- name: Run DafnyTestGeneration Tests
run: dotnet test --no-restore --verbosity normal Source/DafnyTestGeneration.Test
run: dotnet test --no-restore --verbosity normal --logger trx Source/DafnyTestGeneration.Test
- uses: actions/upload-artifact@v2
if: always()
with:
name: unit-test-results-${{ matrix.os }}
path: Source/*/TestResults/*.trx