Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

llvm-cov support #5

Open
gsauthof opened this issue May 16, 2016 · 11 comments
Open

llvm-cov support #5

gsauthof opened this issue May 16, 2016 · 11 comments

Comments

@gsauthof
Copy link

LLVM clang also understands the gcc options for enabling coverage collection (e.g. --coverage and -fprofile-arcs -ftest-coverage). But - in that compatibility mode - llvm targets a certain gcc/gcov version. For example one that isn't supported anymore by current gcov/lcov (yielding inaccuracies and lcov errors).

Thus, to increase the portability of the cmake codecov module, it would be useful to also add support for llvm-cov.

@alehaa
Copy link
Contributor

alehaa commented May 21, 2016

If your compiler is clang >= 3.5, FindGcov should use llvm-cov by default. The llvm-cov-wrapper will be used for compatibility with Lcov.

Can you give me some sample output to debug the situation?

@gsauthof
Copy link
Author

I see, the llvm-cov-wrapper that comes with CMake-codecov calls llvm-cov gcov.
This indeed does work when I compile with clang.

With this issue-ticket I am trying to suggest to add support for LLVM's native coverage mode.

Currently, when I compile with CMake-codecov and clang it is compiled like this:

/usr/bin/clang++     -O0 -g -fprofile-arcs -ftest-coverage ...

In contrast, to use the native mode one would compile like this:

 /usr/bin/clang++     -O0 -g -fprofile-instr-generate -fcoverage-mapping ...

With that mode one also would have to execute different steps to generate the report, e.g.:

llvm-profdata merge default.profraw -o default.profi
llvm-cov show -instr-profile default.profi foo

So this issue is definitely intended as a feature request.

@alehaa
Copy link
Contributor

alehaa commented May 22, 2016

At the moment this CMake module is intended to generate coverage reports for tools which understand gcov data only, like lcov and online tools like codecov.io. I think if the native LLVM coverage format will be enabled, this will break the gcov data which might not be intended, right?

A separate option for compiling for native LLVM coverage format might be added to enable building the LLVM reports. It will be default off, so the normal user will have no problems using this module for gcov data, but interested users might generate LLVM coverage reports. Is this the behavior you're requesting?

@gsauthof
Copy link
Author

Yes, a separate option would be fine.

@rickyzhang82
Copy link

Hi Gentlemen,

Thanks for sharing your cmake file. It saves me a lot of leg work.

It works perfectly in Linux GCC. But in Mac OS X clang, it can't generate HTML report. I can see *.gcda and *.gcno are generated . In Mac OS 10.12.2 XCode 8.2, I found that

ls -al /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
...
lrwxr-xr-x   1 root  wheel     8B Oct 29 09:31 gcov@ -> llvm-cov
-rwxr-xr-x   1 root  wheel   590K Dec  6 18:53 llvm-cov*

gcov is a symlink to llvm-cov. There is no more geninfo and genhtml. But it seems that new llvm-cov can generate HTML report as well in the same command.

I wonder if the CMake-codecov is intended to support Mac OS X? For your debug, here is my CMake output:

-- Found gcov evaluation for Clang: /Users/Ricky/repo/xlparser/cmake/llvm-cov-wrapper
-- Could NOT find lcov (missing:  LCOV_BIN GENINFO_BIN GENHTML_BIN)

@rickyzhang82
Copy link

I dig it deeper. The script llvm-cov-wrapper does try to address the llvm-cov. But it doesn't solve lcov problem. I installed lcov from GNU in Mac OS X to generate report from 'llvm-cov gcov'. GNU lcov doesn't seem to be compatible with llvm-cov.

I tried gcovr, a Python implementation report coverage generator. The detailed result doesn't match.

So IMHO I need to use llvm-cov native mode and generate report by 'llvm-cov show --format=html'

/usr/bin/clang++     -O0 -g -fprofile-instr-generate -fcoverage-mapping ...

@alehaa
Copy link
Contributor

alehaa commented Dec 17, 2016

Ok. As far as I understand we need a new mode for the codecov scripts to handle a) native llvm-cov and b) exporting the llvm-cov reports as HTML instead of using lcov. Right?

@rickyzhang82
Copy link

I'm working on it now.

@rickyzhang82
Copy link

No, it turns out that Mac OS X llvm-cov is not the same as Clang llvm-cov. It doesn't support -format argument. It still can't show html. Any suggestions?

@alehaa
Copy link
Contributor

alehaa commented Dec 17, 2016

I'll try to generalize the modules, so we can plug different analyzing-tools. Then we can investigate how to generate the reports for llvm-cov in XCode. ;)

@rickyzhang82
Copy link

Sorry, I don't follow your plan. Can you be more specific on "the modules" and "different analyzing-tools"?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

3 participants