List the tests that exercise a function

[smoelius]

When someone fixes a piece of code, it is considered good practice to write a test that exercises the fix.

In many cases, an existing test could be modified to save having to write a completely new test from scratch.

How difficult do you think it would be for a tool like cargo-llvm-cov to list the tests that exercise a function?

[taiki-e]

I don't know a good way to do this.

I guess the simplest, stupidest, and accurate way would be to run all the tests individually and analyze the resulting JSON reports to see if they affect the coverage of the target function, but that would take too long.

If listing is not your end goal, you might be able to use the feature used in coverage guided fuzzing, but I don't know how that works.

[smoelius]

If listing is not your end goal, you might be able to use the feature used in coverage guided fuzzing, but I don't know how that works.

What are you referring to?

[taiki-e]

What are you referring to?

I was thinking about SanitizerCoverage used in LibFuzzer. However, this may not be a match for the use case here, as this seem more suited to changing the input and getting it's impact on the coverage.

https://www.llvm.org/docs/LibFuzzer.html
https://clang.llvm.org/docs/SanitizerCoverage.html

[smoelius]

Thanks very much for your responses.

I am sensing that having the ability to list the tests that exercise a function doesn't sound valuable to you?

[taiki-e]

I am sensing that having the ability to list the tests that exercise a function doesn't sound valuable to you?

To be honest, it depends on what the implementation is. If it is relatively easy to implement as an extension of the current cargo-llvm-cov, then I think it is reasonable to implement it as part of the cargo-llvm-cov. Otherwise, it would make more sense to implement it as an individual tool that uses the output of the cargo-llvm-cov, like the llvm-cov-pretty.