Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unstable mutation coverage percentages #52

Closed
vmassol opened this issue Mar 27, 2018 · 3 comments
Closed

Unstable mutation coverage percentages #52

vmassol opened this issue Mar 27, 2018 · 3 comments

Comments

@vmassol
Copy link
Contributor

vmassol commented Mar 27, 2018

When I run pitest/descartes on some xwiki modules, I get different percentage values across runs. for example I had 69% of xwiki-commons-crypto-common and I've set the build to fail if it gets below that values.

Without making any change to that module when I ran it today, the build failed with:

xwiki-commons-crypto-common: Mutation score of 68 is below threshold of 69

It's important that the number be stable. Any idea what could cause this?

@oscarlvp
Copy link
Member

Can you share the report files? It would be nice to have them in JSON or XML format to run some analysis and verify this issue.

@vmassol
Copy link
Contributor Author

vmassol commented Mar 27, 2018

The problem is that I don't have the report generating the score of 68 anymore. Here's a zip with the pitest output for the 69 threshold: https://up1.xwikisas.com/#uv-_aed_swRVwHfFD3iRmg

Thx

@oscarlvp
Copy link
Member

The discussion about this issue has been ported to #62

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants