Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add system for evaluating coverage of created assertions #52

Closed
selkamand opened this issue Mar 5, 2023 · 1 comment · Fixed by #53
Closed

Add system for evaluating coverage of created assertions #52

selkamand opened this issue Mar 5, 2023 · 1 comment · Fixed by #53

Comments

@selkamand
Copy link
Owner

see #26 for context.

We should build

  1. utilities for extracting the names of all assertion functions in our codebase
  2. utilities for counting the number of unit tests (i.e. expect_ statements that test each assertion)
  3. a system that produces a badge describing (proportion of assertions with > some_threshold unit tests)

Some potential options for badges here:
basically we just need to produce a json file somewhere in this repo that we could point to.

https://css-tricks.com/adding-custom-github-badges-to-your-repo/

@selkamand
Copy link
Owner Author

For now I think we ditch the badge idea.

Instead, I've added a unit test that only passes if all assert_ functions have at least 5 unit tests (expect statements).

Caveats:

  1. assert_create and assert_create_chain are not included, but already quite well covered
  2. detection of unit tests is a little rough. It only detects them if the expect statement and assert_ function are in the same line

Now the only way to pass unit tests is to have well tested code. Unfortunately its currently based on # of unit tests rather than code coverage.

Longer term, if covr or other solutions become able to calculate coverage of functions created upon loading of a package, that would supplement this solution nicely

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant