Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tracking CI correctness #246

Open
qinsoon opened this issue Sep 19, 2023 · 0 comments
Open

Tracking CI correctness #246

qinsoon opened this issue Sep 19, 2023 · 0 comments

Comments

@qinsoon
Copy link
Member

qinsoon commented Sep 19, 2023 •
edited
Loading

Based on the results from #211, created from ci-expected-results.yml introduced in the PR. We may still have random failures that we haven't noticed.

We should create separate issues if we notice any failure, and add a link to the issues in the following table.

linux-x64

Test Name SemiSpace (fastdebug) SemiSpace (release) GenCopy (fastdebug) GenCopy (release) Immix (fastdebug) Immix (release) GenImmix (fastdebug) GenImmix (release) StickyImmix (fastdebug) StickyImmix (release) MarkSweep (fastdebug) MarkSweep (release) MarkCompact (fastdebug) MarkCompact (release)
avrora ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
batik ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
biojava ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
cassandra ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❓ ❓ ❓ ✅
eclipse ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❓ ❓ ✅ ✅
fop ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❓ ❓ ✅ ✅
graphchi ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
h2 ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❓ ❓ ✅ ✅
h2o ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❓ #247 ✅ ✅
jme ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
jython ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❓ ❓ ✅ ✅
kafka ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❓ #248 ✅
luindex ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
lusearch ✅ ✅ ✅ ✅ ✅ ✅ ❓ ✅ ✅ ✅ ✅ ✅ ✅ ✅
pmd ✅ ✅ ✅ ✅ ✅ ✅ ❓ #262 ✅ ❓ #262 ✅ ✅ ✅ ✅ ✅
sunflow ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
tomcat ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❓ ✅ ✅ ✅ ✅ ✅ ✅
xalan ✅ ❓ ✅ ❓ ✅ ✅ ✅ ❓ #245 ❓ ✅ ✅ ✅ ✅ ✅
zxing ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
@qinsoon qinsoon mentioned this issue Sep 19, 2023
Merged
caizixian pushed a commit that referenced this issue Sep 20, 2023
@qinsoon
Check results for new CI, allow some benchmarks to fail (#211)
0e04c17 
This PR adds a yaml file that records our expected results for DaCapo
tests. It also adds a script to check the results for each CI run. We
should generally expect a CI run to succeed (which means the results are
expected) once this PR is merged.

However, as we may have random failures that we haven't noticed, it is
possible a CI run still fails. In such a case, we should update the
`ci-expected-results.yml`, create an Github issue, and update
#246.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@qinsoon