Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mutation testing - logger tracking missing/timeout/unviable tests #4179

Closed
wants to merge 13 commits into from

Conversation

ASuciuX
Copy link
Contributor

@ASuciuX ASuciuX commented Dec 15, 2023

Applicable issues

Additional info (benefits, drawbacks, caveats)

Solutions and Recommended Usage

Cargo-mutant does an incremental build for each mutation, that’s why this solution comes with the implementation of tracking the output and only updating it with the differences, to run 10-20 mutants instead of 4000.

CI YML File: Are linked with the actions in the actions' repo stacks-network/actions#3.

Should be executed with every PR, as it tests only the updated or newly added functions in the PR's commits.

Handling Exit Codes (docs)

In the logger workflow they are updating the place they belong to.

Checklist

  • Fix mutation testing on packages to run on the supposed functions, not just on some
  • PR CI checker to have all functions tested
    • missed/unviable/timeout outputs result in an error and stop the PR from being merged
    • outputs the functions that have to be tested
  • On merge on main/develop/next branch override (by appending) current output with the merged one
  • Mutation Testing Coverage for the already existing functions
    • Mutation testing triggering on git diff that didn't affect the functions, but affected the tests. (if the function already exists and is not modified and the tests that use it are created/updated, the functions will not be included in the cargo mutants)
  • Mutants initial sync completed for develop and next branches

@ASuciuX ASuciuX self-assigned this Dec 15, 2023
@ASuciuX ASuciuX changed the title Test/mutants logger Mutation testing - logger tracking missing/timeout/unviable tests Dec 15, 2023
Copy link

codecov bot commented Dec 15, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (1f5cc3f) 62.42% compared to head (53cc9c4) 82.61%.
Report is 20 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff              @@
##           develop    #4179       +/-   ##
============================================
+ Coverage    62.42%   82.61%   +20.18%     
============================================
  Files          401      401               
  Lines       286383   288405     +2022     
============================================
+ Hits        178775   238256    +59481     
+ Misses      107608    50149    -57459     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@obycode
Copy link
Contributor

obycode commented Dec 19, 2023

Feedback from PR meeting - let's move to draft for now.

@obycode obycode marked this pull request as draft December 19, 2023 15:26
@obycode
Copy link
Contributor

obycode commented Dec 19, 2023

Related discussion in Nakamoto slack channel.

@saralab saralab requested review from wileyj and kantai January 16, 2024 15:25
@saralab saralab marked this pull request as ready for review January 16, 2024 21:37
# test
steps:
- name: Run logging mutants from actions
uses: stacks-network/actions/mutation-testing/logger@feat/mutation-testing
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this workflow doesn't seem to exist on this branch (or in main). is that something that needs to be re-added?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend merging the feat/mutation-testing branch into stacks-network/actions first, so you don't have to make another PR changing/removing the branch name later

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I can't find the logger action either

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I can't find the logger action either

i expect this PR will be changing quite a bit as we've just started testing this. i'm converting this to draft for now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, i will be updating it these days, wanted to have the PR-differences fully finished asap. this one will have to wait till we have all the mutants from the general sync before being merged

@saralab saralab requested a review from jbencin January 23, 2024 15:31
@wileyj wileyj marked this pull request as draft January 24, 2024 20:36
@ASuciuX ASuciuX closed this Apr 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants