- Introduction
- Motivation
- Getting Started
- Implementation
- Issues with GCC < 10 on Linux
This package consists of a set of CMake scripts that download and compile libclang into a single static archive containing all LLVM and third party dependencies so applications which link against it can be easily deployed. The build also produces a bundled shared library which allows faster iteration during development time by avoiding link time slowdowns. The build takes only about 7 minutes on my modestly provisioned 7 year old i5/16GB RAM Thinkpad instead of 5-7 hours for a normal from-scratch LLVM build.
I tested it on Manjaro Linux with GCC 11.1.0, macOS Mojave and Windows 10 using
the MS Visual C++ toolchain. As an aside the Windows build is only possible
because the Zig project generously provides prebuilt statically linked LLVM
libraries for Windows, if you are benefiting please consider contributing, it’s
immensely annoying and time-consuming to build LLVM and clang from scratch
and we should support Andy Kelley for saving us the trouble.
A small easy-to-build demo app clang_visitor that parses some C++ is also
included to help you get started. And to convince you the build works as
advertised below are the minimal runtime dependencies on each of the supported
platforms when the app is linked against the static archive:
- Linux
> $ ldd ./clang_visitor linux-vdso.so.1 (0x00007ffce8bfb000) libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007fdb9954a000) libm.so.6 => /usr/lib/libm.so.6 (0x00007fdb99404000) libdl.so.2 => /usr/lib/libdl.so.2 (0x00007fdb993ff000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fdb993dd000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007fdb993c3000) libc.so.6 => /usr/lib/libc.so.6 (0x00007fdb991fd000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fdb9ea9b000)
- macOS Mojave:
> otool -L clang_visitor clang_visitor: /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11) /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.4) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
- … and Windows 10
> dumpbin.exe /DEPENDENTS clang_visitor.exe Microsoft (R) COFF/PE Dumper Version 14.25.28614.0 Copyright (C) Microsoft Corporation. All rights reserved. Dump of file clang_visitor_static.exe File Type: EXECUTABLE IMAGE Image has the following dependencies: VERSION.dll KERNEL32.dll SHELL32.dll ole32.dll OLEAUT32.dll ADVAPI32.dll VCRUNTIME140.dll VCRUNTIME140_1.dll api-ms-win-crt-stdio-l1-1-0.dll api-ms-win-crt-runtime-l1-1-0.dll api-ms-win-crt-heap-l1-1-0.dll api-ms-win-crt-utility-l1-1-0.dll api-ms-win-crt-environment-l1-1-0.dll api-ms-win-crt-string-l1-1-0.dll api-ms-win-crt-convert-l1-1-0.dll api-ms-win-crt-time-l1-1-0.dll api-ms-win-crt-math-l1-1-0.dll api-ms-win-crt-locale-l1-1-0.dll api-ms-win-crt-filesystem-l1-1-0.dll Summary 2C4000 .data 123000 .pdata 15CB000 .rdata 7D000 .reloc 1000 .rsrc 2105000 .text
Currently the best way to statically analyze C and C++ source is libclang.
Unfortunately applications built against libclang aren’t very portable or easy
to deploy because of dependencies on third party libraries like ncurses and z3
and the libclang shared library itself. Package managers do a decent job of
orchestrating the install but it’s still hard to deploy an application that’s
pinned to a specific version of libclang or to ship binaries between Linux
distributions. There’s always containers or Nix or Guix but I think asking
people to get up to speed on purely functional package managers or have Docker
running just to use libclang apps is a non-starter. With this package all you
need is CMake on macOS and Linux and additionally Visual Studio Build Tools on
Windows.
Below are some instructions on getting up and running on Linux, macOS and Windows 10. Everything beyond that is the full build as a literate program and only interesting if you care about implementation details. If you just want to use this package it can be safely skipped. Enjoy!
First make sure you have a cmake version greater that 3.13:
> cmake --version cmake version 3.17.0 CMake suite maintained and supported by Kitware (kitware.com/cmake).
and gcc~/~g++ > 10:
> g++ --version g++ (Debian 10.2.1-6) 10.2.1 20210110 Copyright (C) 2020 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Clone this repo, create a build directory inside it and run the build and install:
> git clone https://github.com/deech/libclang-static-build > cd libclang-static-build > mkdir build; cd build > cmake .. -DCMAKE_INSTALL_PREFIX=.. > make install
The install step copies all the artifacts to the directory into which you
cloned this repo just above the build directory. Nothing else on the system is
touched.
Once it’s done installing there will be 3 new directories in repo directory,
lib, include and share. The first contains a big libclang static archive
with all dependencies bundled and shared versions of those libraries for quicker
compilation during development, the second contains the libclang headers and
the third has two directories share/doc/examples/static and
share/doc/examples/shared both of which contain a couple of identical small
examples that shows how to create static and shared libclang apps.
The two example directories share/doc/examples/static and
share/doc/examples/shared both of which contain an identical small example
program that walks a C++ header file containing an enum, the first builds a
large (100MB+) statically linked executable with minimal dependencies and the
second a much smaller shared executable which depends libclang, ncurses and
z3 at runtime. To build them run make in their respective directories:
> cd libclang-static-build > cd doc/example/static > make > ./clang_visitor Cursor spelling, kind: __ENUM__, macro definition Cursor spelling, kind: Enum, EnumDecl Cursor spelling, kind: RED, EnumConstantDecl Cursor spelling, kind: , UnexposedExpr Cursor spelling, kind: , IntegerLiteral Cursor spelling, kind: , IntegerLiteral Cursor spelling, kind: GREEN, EnumConstantDecl Cursor spelling, kind: , UnexposedExpr Cursor spelling, kind: , BinaryOperator Cursor spelling, kind: , BinaryOperator Cursor spelling, kind: , IntegerLiteral Cursor spelling, kind: , IntegerLiteral Cursor spelling, kind: BLUE, EnumConstantDecl Cursor spelling, kind: , UnexposedExpr Cursor spelling, kind: , BinaryOperator Cursor spelling, kind: , BinaryOperator Cursor spelling, kind: RED, DeclRefExpr Cursor spelling, kind: GREEN, DeclRefExpr
First install CMake and Build Tools For Visual Studio 2019, then clone this
repo, create a build directory inside it, run the build and install:
> git.exe clone https://github.com/deech/libclang-static-build > cd libclang-static-build > mkdir build > cd build > cmake.exe .. -Thost=x64 -G "Visual Studio 16 2019" -A x64 -DCMAKE_INSTALL_PREFIX=.. -DCMAKE_BUILD_TYPE=Release -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD="AVR" -DLLVM_ENABLE_LIBXML2=OFF -DLLVM_USE_CRT_RELEASE=MT > "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Current\Bin\MSBuild.exe" /m -p:Configuration=Release INSTALL.vcxproj
At the final step I needed to give the full path to MSBuild.exe even though I
asked MS Build Tools to add it to the PATH so I reproduced it here so you
don’t have to go hunt it down.
There should now be 3 new directories in the repo directory, lib, include,
and share. The first contains clang_static_bundled.lib which is a 400MB
static archive, include has all the headers needed to build libclang apps
and share has a single statically linked demo app.
The example directory share/doc/examples/static contains a small demo
that parses out a C++ enum from a header file. To build it:
> cd libclang-static-build\share\doc\examples\static > mkdir build > cd build > cmake.exe -G "Visual Studio 16 2019" .. -DCMAKE_INSTALL_PREFIX=.. > "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Current\Bin\MSBuild.exe" /m -p:Configuration=Release INSTALL.vcxproj
The directory above the build directory now contains bin which contains the
example app clang_visitor.exe:
>cd ..\bin >clang_visitor_static.exe Cursor spelling, kind: __ENUM__, macro definition Cursor spelling, kind: Enum, EnumDecl Cursor spelling, kind: RED, EnumConstantDecl Cursor spelling, kind: , IntegerLiteral Cursor spelling, kind: GREEN, EnumConstantDecl Cursor spelling, kind: , BinaryOperator Cursor spelling, kind: , IntegerLiteral Cursor spelling, kind: , IntegerLiteral Cursor spelling, kind: BLUE, EnumConstantDecl Cursor spelling, kind: , BinaryOperator Cursor spelling, kind: RED, DeclRefExpr Cursor spelling, kind: GREEN, DeclRefExpr
At a high level to build a bundled shared and static library I grab the
prebuilt clang+LLVM static archives and libclang sources, build the latter
from scratch locally and then bundle it along with all the prebuilt archives
into one large library that an executable can link against.
Additionally on Linux and macOS libclang depends z3 and ncurses. While the
former has official prebuilt releases the latter does not and so we have to
build from source locally. Both are then folded into the resulting library.
On Windows 10 the situation is nicer because the Zig project provides prebuilt
LLVM archives with no dependency on z3 so the build actually goes quite a bit
faster than the other platforms. Do support Zig if you can.
And finally we generate a small demo that traverses a C++ header; on Linux and macOS it’s a standard Make project and a CMake project on Windows.
cmake_minimum_required(VERSION 3.13)
project(libclang-static-build)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake/modules")
set(LIBCLANG_EXAMPLES "${CMAKE_CURRENT_SOURCE_DIR}/cmake/examples")
if(NOT (MSVC OR APPLE OR UNIX))
message(FATAL_ERROR "This build currenly works only with macOS, Microsoft Visual Studio and Linux.")
endif()
if(APPLE OR UNIX)
find_program(CMAKE_LIBTOOL libtool)
if(NOT CMAKE_LIBTOOL)
message(FATAL_ERROR "'libtool' is necessary for building static archives")
endif()
include(LinuxMacosBuild)
else()
include(MSVCBuild)
endif()“Reproducibility” is achieved by hard-coding the URLs from which to get the dependencies, I’m sure there’s more principled ways but this works ok for now.
if(APPLE)
set(LIBCLANG_PREBUILT_URL https://github.com/llvm/llvm-project/releases/download/llvmorg-13.0.0/clang+llvm-13.0.0-x86_64-apple-darwin.tar.xz)
else()
set(LIBCLANG_PREBUILT_URL https://github.com/llvm/llvm-project/releases/download/llvmorg-13.0.0/clang+llvm-13.0.0-x86_64-linux-gnu-ubuntu-20.04.tar.xz)
endif()
set(CLANG_SOURCES_URL https://github.com/llvm/llvm-project/releases/download/llvmorg-13.0.0/clang-13.0.0.src.tar.xz)
set(NCURSES_SOURCES_URL https://ftp.gnu.org/pub/gnu/ncurses/ncurses-6.2.tar.gz)
if(APPLE)
set(Z3_PREBUILT_URL https://github.com/Z3Prover/z3/releases/download/z3-4.8.7/z3-4.8.7-x64-osx-10.14.6.zip)
else()
set(Z3_PREBUILT_URL https://github.com/Z3Prover/z3/releases/download/z3-4.8.7/z3-4.8.7-x64-ubuntu-16.04.zip)
endif()The dependencies are then downloaded and unpacked at build time …
include(Download)
message(STATUS "Downloading ncurses sources, prebuilt z3 & prebuilt libclang with sources; this is ~500MB, please be patient, 'libclang_prebuilt' will take several minutes ...")
set(NCURSES_SOURCE_DIR)
download(ncurses_sources ${NCURSES_SOURCES_URL} NCURSES_DOWNLOAD_DIR)
set(LIBCLANG_SOURCES_DIR)
download(clang_sources ${CLANG_SOURCES_URL} LIBCLANG_SOURCES_DIR)
set(Z3_PREBUILT_DIR)
download(z3_prebuilt ${Z3_PREBUILT_URL} Z3_PREBUILT_DIR)
set(LIBCLANG_PREBUILT_DIR)
download(libclang_prebuilt ${LIBCLANG_PREBUILT_URL} LIBCLANG_PREBUILT_DIR)Since ncurses does not provide prebuilt static archives we build it locally
using a recipe stolen from Arch scripts:
include(ExternalProject)
ExternalProject_Add(ncurses
SOURCE_DIR ${NCURSES_DOWNLOAD_DIR}
CONFIGURE_COMMAND <SOURCE_DIR>/configure --enable-rpath --prefix=${CMAKE_INSTALL_PREFIX} --with-shared --with-static --with-normal --without-debug --without-ada --enable-widec --disable-pc-files --with-cxx-binding --without-cxx-shared --with-abi-version=5
BUILD_COMMAND make
INSTALL_COMMAND ""
)This part is why I used CMake for this project in the first place, the LLVM
project provides CMake scripts that contain useful functions and macros which
take care of the nitty gritty C++ compiler and inclusion flags that allow
building libclang from source, without them this project would have been
impossible.
list(APPEND CMAKE_MODULE_PATH "${LIBCLANG_PREBUILT_DIR}/lib/cmake/clang")
list(APPEND CMAKE_MODULE_PATH "${LIBCLANG_PREBUILT_DIR}/lib/cmake/llvm")
list(APPEND CMAKE_MODULE_PATH "${LIBCLANG_SOURCES_DIR}/cmake/modules")
include(LibClangBuild)
include(HandleLLVMOptions)
include(AddLLVM)
include(AddClang)
include(GatherArchives)For some reason macOS needs to be told to use C++14 and it doesn’t hurt to
include it for Linux as well:
set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-rtti")get_libclang_sources_and_headers populates the last three arguments with
absolute paths to headers, libclang sources and the included LLVM archives.
get_libclang_sources_and_headers(
${LIBCLANG_SOURCES_DIR}
${LIBCLANG_PREBUILT_DIR}
LIBCLANG_SOURCES
LIBCLANG_ADDITIONAL_HEADERS
LIBCLANG_PREBUILT_LIBS
)Now we have to tell the CMake recipe where to find the shared libraries for the z3 and ncurses dependencies:
include_directories(${LIBCLANG_PREBUILT_DIR}/include)
ExternalProject_Get_Property(ncurses BINARY_DIR)
set(NCURSES_BINARY_DIR ${BINARY_DIR})
set(NCURSES_SHARED_LIB)
if(APPLE)
set(NCURSES_SHARED_LIB ${NCURSES_BINARY_DIR}/lib/libncursesw.dylib ${NCURSES_BINARY_DIR}/lib/libncursesw.5.dylib)
else()
set(NCURSES_SHARED_LIB ${NCURSES_BINARY_DIR}/lib/libncursesw.so ${NCURSES_BINARY_DIR}/lib/libncursesw.so.5 ${NCURSES_BINARY_DIR}/lib/libncursesw.so.5.9)
endif()
unset(BINARY_DIR)
if(APPLE)
set(Z3_SHARED_LIB ${Z3_PREBUILT_DIR}/bin/libz3.dylib)
else()
set(Z3_SHARED_LIB ${Z3_PREBUILT_DIR}/bin/libz3.so)
endif()add_clang_library is a libclang provided CMake function that does all the
hard work of generating Makefiles to build a clang and LLVM based library or
executable. It’s used twice, once to generate a static archive and once more for
a shared library. I’m building it twice because building with both SHARED and
STATIC in a single call seems to produce objects compiled with -fPIC so
building the shared library fails. I’m probably doing something wrong but this
seems to work for now.
add_clang_library(libclang
SHARED
OUTPUT_NAME clang
${LIBCLANG_SOURCES}
ADDITIONAL_HEADERS ${LIBCLANG_ADDITIONAL_HEADERS}
LINK_LIBS
${LIBCLANG_PREBUILT_LIBS} ${NCURSES_SHARED_LIB} dl pthread z
LINK_COMPONENTS ${LLVM_TARGETS_TO_BUILD}
DEPENDS ncurses
)
add_clang_library(libclang_static
STATIC
OUTPUT_NAME clang_static
${LIBCLANG_SOURCES}
ADDITIONAL_HEADERS ${LIBCLANG_ADDITIONAL_HEADERS}
DEPENDS ncurses
)
set_target_properties(libclang PROPERTIES VERSION 12)This bit is pretty much copy-pasta’ed from the CMake build scripts that come
with clang sources probably doesn’t do much.
if(APPLE)
set(LIBCLANG_LINK_FLAGS " -Wl,-compatibility_version -Wl,1")
set_property(TARGET libclang APPEND_STRING PROPERTY
LINK_FLAGS ${LIBCLANG_LINK_FLAGS})
else()
set_target_properties(libclang
PROPERTIES
DEFINE_SYMBOL _CINDEX_LIB_)
endif()On MacOS libtool can reliably nest multiple archives into one by simply
passing them in as arguments.
Unfortunately Linux is more complicated. Due to limitations of ar I have to
make a thin archive, a static archive which doesn’t actually contain the other
archives but references them, by calling ar with the T (for thin) argument.
Then I copy all archives I need to bundle into one directory
ALL_ARCHIVES_DIRECTORY and build the thin archive there because a thin archive
finds referenced archives by where they were relative to it when it was created.
At install time the thin archives are the referenced archives are copied the
same final location so the relative paths are intact and apps can transparently
link with the thin archive.
The call to gather_archives populates ALL_ARCHIVES_DIRECTORY with a
hard-coded path local to the CMake build directory, ALL_ARCHIVE_NAMES with a
list of the archives basenames (just the filename without parents) so they can
be passed to ar and ALL_ARCHIVE_PATHS with a list of archives with their
full paths which is used at install time to copy them to the same location. This
is not good code.
if(APPLE)
add_custom_target(
libclang_bundled ALL
COMMAND ${CMAKE_LIBTOOL} -static -o libclang_bundled.a
${CMAKE_CURRENT_BINARY_DIR}/libclang_static.a
${LIBCLANG_PREBUILT_LIBS}
${NCURSES_BINARY_DIR}/lib/libncursesw.a
${Z3_PREBUILT_DIR}/bin/libz3.a
DEPENDS ncurses libclang libclang_static
)
else()
gatherArchives(
ALL_ARCHIVES_DIRECTORY
ALL_ARCHIVE_NAMES
ALL_ARCHIVE_PATHS
${CMAKE_CURRENT_BINARY_DIR}/libclang_static.a
${LIBCLANG_PREBUILT_LIBS}
${NCURSES_BINARY_DIR}/lib/libncursesw.a
${Z3_PREBUILT_DIR}/bin/libz3.a
)
add_custom_target(
gather_archives ALL
COMMAND ${CMAKE_COMMAND} -E make_directory ${ALL_ARCHIVES_DIRECTORY}
COMMAND ${CMAKE_COMMAND} -E copy
${CMAKE_CURRENT_BINARY_DIR}/libclang_static.a
${LIBCLANG_PREBUILT_LIBS}
${NCURSES_BINARY_DIR}/lib/libncursesw.a
${Z3_PREBUILT_DIR}/bin/libz3.a
${ALL_ARCHIVES_DIRECTORY}
DEPENDS ncurses libclang libclang_static
)
add_custom_target(
libclang_bundled ALL
COMMAND ${CMAKE_AR} crsT libclang_bundled.a ${ALL_ARCHIVE_NAMES}
WORKING_DIRECTORY ${ALL_ARCHIVES_DIRECTORY}
DEPENDS gather_archives
)
endif()Now it’s time to create the example app, Makefiles are generated using the
Makefile templates covered in Static Makefile and Shared Makefile and they are
all copied into an overall examples directory in CMAKE_CURRENT_BINARY_DIR,
the CMake build directory.
That CMAKE_OSX_SYSROOT thing is simply so libclang headers can find the
time.h on macOS. I’m really not sure why it isn’t in the standard location.
set(MAKEFILE_LIBCLANG_INCLUDE ${CMAKE_INSTALL_PREFIX}/include)
if(APPLE)
set(MAKEFILE_LIBCLANG_INCLUDE "${MAKEFILE_LIBCLANG_INCLUDE} -I${CMAKE_OSX_SYSROOT}/usr/include")
endif()
set(MAKEFILE_LIBCLANG_LIBDIR ${CMAKE_INSTALL_PREFIX}/lib)
file(MAKE_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/examples/static)
configure_file(${LIBCLANG_EXAMPLES}/Makefile_static.in ${CMAKE_CURRENT_BINARY_DIR}/examples/static/Makefile)
if(APPLE)
configure_file(${LIBCLANG_EXAMPLES}/Makefile_shared_macos.in ${CMAKE_CURRENT_BINARY_DIR}/examples/shared/Makefile)
else()
configure_file(${LIBCLANG_EXAMPLES}/Makefile_shared.in ${CMAKE_CURRENT_BINARY_DIR}/examples/shared/Makefile)
endif()
file(COPY ${LIBCLANG_EXAMPLES}/clang_visitor.c DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/examples/static)
file(COPY ${LIBCLANG_EXAMPLES}/sample.H DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/examples/static)
file(COPY ${LIBCLANG_EXAMPLES}/clang_visitor.c DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/examples/shared)
file(COPY ${LIBCLANG_EXAMPLES}/sample.H DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/examples/shared)Now everything is installed relative to the CMAKE_INSTALL_PREFIX, the
libraries under lib, clang headers under include/clang-c and examples
under share/doc.
if(APPLE)
set(LIBCLANG_INSTALL_LIBS
${CMAKE_CURRENT_BINARY_DIR}/libclang_bundled.a
${Z3_PREBUILT_DIR}/bin/libz3.a
${Z3_SHARED_LIB}
${NCURSES_BINARY_DIR}/lib/libncursesw.a
${NCURSES_SHARED_LIB}
)
else()
set(LIBCLANG_INSTALL_LIBS
${ALL_ARCHIVES_DIRECTORY}/libclang_bundled.a
${ALL_ARCHIVE_PATHS}
${Z3_SHARED_LIB}
${NCURSES_SHARED_LIB}
)
endif()
install(PROGRAMS ${LIBCLANG_INSTALL_LIBS} DESTINATION lib)
install(DIRECTORY ${LIBCLANG_PREBUILT_DIR}/include/clang-c DESTINATION include)
install(DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/examples DESTINATION share/doc)The Windows build has the same overall idea as Linux/macOS it’s just different enough that it’s easier to start over than share code.
The initial bits are similar, this time the prebuilt LLVM is downloaded from the
Zig project and it has all the dependencies built in so nothing more is needed
except the clang sources.
set(LIBCLANG_PREBUILT_URL https://ziglang.org/deps/llvm+clang+lld-13.0.0-x86_64-windows-msvc-release-mt.tar.xz)
set(CLANG_SOURCES_URL https://github.com/llvm/llvm-project/releases/download/llvmorg-13.0.0/clang-13.0.0.src.tar.xz)
include(Download)
message(STATUS "Downloading prebuilt libclang with sources; this is ~500MB, please be patient, 'libclang_prebuilt' will take several minutes ...")
download(clang_sources ${CLANG_SOURCES_URL} LIBCLANG_SOURCES_DIR)
download(libclang_prebuilt ${LIBCLANG_PREBUILT_URL} LIBCLANG_PREBUILT_DIR)Then I pull the the LLVM CMake scripts into scope just as before
list(APPEND CMAKE_MODULE_PATH "${LIBCLANG_PREBUILT_DIR}/lib/cmake/clang")
list(APPEND CMAKE_MODULE_PATH "${LIBCLANG_PREBUILT_DIR}/lib/cmake/llvm")
list(APPEND CMAKE_MODULE_PATH "${LIBCLANG_SOURCES_DIR}/cmake/modules")
set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
include(LibClangBuild)
include(HandleLLVMOptions)
include(AddLLVM)
include(AddClang)
get_libclang_sources_and_headers(
${LIBCLANG_SOURCES_DIR}
${LIBCLANG_PREBUILT_DIR}
LIBCLANG_SOURCES
LIBCLANG_ADDITIONAL_HEADERS
LIBCLANG_PREBUILT_LIBS
)
include_directories(${LIBCLANG_PREBUILT_DIR}/include)
add_clang_library(libclang
SHARED
STATIC
OUTPUT_NAME clang
${LIBCLANG_SOURCES}
LINK_LIBS ${LIBCLANG_PREBUILT_LIBS} Version
ADDITIONAL_HEADERS ${LIBCLANG_ADDITIONAL_HEADERS}
)
set_target_properties(libclang PROPERTIES VERSION 12)This bit is important, without it every object file spews a inconsistent DLL
linkage warning. More importantly for reasons I don’t understand, I have to do
this as opposed to how the LLVM project does it: set_target_properties(libclang
PROPERTIES DEFINE_SYMBOL _CINDEX_LIB_)
target_compile_definitions(obj.libclang PUBLIC "_CINDEX_LIB_")While the clang build function provided by LLVM produces a static library
called clang_static.lib any app that links against it also requires
libclang.dll at runtime, so we have just created a 400MB static library that
doesn’t do anything. I guess static libs that also carry a dependency on DLLs is
a common idiom on Windows but defeats the purpose of this project.
However I found an intermediate static archive obj.libclang.lib that also
gets generated and does what I want so I pass that to the archive tool instead.
I’m sure there is a better way but this seems to work for now.
LIBCLANG_INSTALL_LIBS is used at install time to copy the fat archive to the
right place.
find_program(lib_tool lib)
if(NOT lib_tool)
get_filename_component(CXX_COMPILER_DIRECTORY "${CMAKE_CXX_COMPILER}" PATH)
set(lib_tool "${CXX_COMPILER_DIRECTORY}/lib.exe")
endif()
set(AR_COMMAND ${lib_tool} /NOLOGO /OUT:${CMAKE_CURRENT_BINARY_DIR}/clang_static_bundled.lib "${CMAKE_CURRENT_BINARY_DIR}/obj.libclang.dir/Release/obj.libclang.lib" ${LIBCLANG_PREBUILT_LIBS})
add_custom_target(libclang_static_bundled ALL
COMMAND ${AR_COMMAND}
DEPENDS libclang
BYPRODUCTS ${CMAKE_CURRENT_BINARY_DIR}/clang_static_bundled.lib
)
set(LIBCLANG_INSTALL_LIBS ${CMAKE_CURRENT_BINARY_DIR}/clang_static_bundled.lib)And now for the example demo, I fill out the template CMake build recipe, copy
the sources and build scripts to an examples directory in the local build
directory …
set(CMAKE_MSVC_LIB_DIR ${CMAKE_INSTALL_PREFIX}/lib)
set(CMAKE_MSVC_INCLUDE_DIR ${CMAKE_INSTALL_PREFIX}/include)
configure_file(${LIBCLANG_EXAMPLES}/CMakeLists.MSVC.in ${CMAKE_CURRENT_BINARY_DIR}/examples/static/CMakeLists.txt)
file(COPY ${LIBCLANG_EXAMPLES}/sample.H DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/examples/static/bin)
file(COPY ${LIBCLANG_EXAMPLES}/clang_visitor.c DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/examples/static)
file(COPY ${LIBCLANG_EXAMPLES}/README.txt DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/examples/static)… and install the library, clang headers and demos under
CMAKE_INSTALL_PREFIX\{lib,include,share} respectively
install(PROGRAMS ${LIBCLANG_INSTALL_LIBS} DESTINATION lib)
install(DIRECTORY ${LIBCLANG_PREBUILT_DIR}/include/clang-c DESTINATION include)
install(DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/examples DESTINATION share/doc)include(FetchContent)
function (download name url source_dir)
FetchContent_Declare(${name} URL ${url})
if(NOT ${name}_POPULATED)
message(STATUS "* Downloading ${name} from ${url}")
FetchContent_Populate(${name})
endif()
set(${source_dir} ${${name}_SOURCE_DIR} PARENT_SCOPE)
endfunction()These are the clang source files and headers needed to build libclang, most
have been copied wholesale from the CMakeLists.txt provided with the project.
set(LIBCLANG_SOURCE_PATH tools/libclang)
set(LIBCLANG_INCLUDE_PATH include/clang-c)
set(LIBCLANG_SOURCE_FILES
ARCMigrate.cpp
BuildSystem.cpp
CIndex.cpp
CIndexCXX.cpp
CIndexCodeCompletion.cpp
CIndexDiagnostic.cpp
CIndexHigh.cpp
CIndexInclusionStack.cpp
CIndexUSRs.cpp
CIndexer.cpp
CXComment.cpp
CXCursor.cpp
CXIndexDataConsumer.cpp
CXCompilationDatabase.cpp
CXLoadedDiagnostic.cpp
CXSourceLocation.cpp
CXStoredDiagnostic.cpp
CXString.cpp
CXType.cpp
Indexing.cpp
FatalErrorHandler.cpp
)
set(LIBCLANG_ADDITIONAL_HEADER_FILES
CIndexDiagnostic.h
CIndexer.h
CXCursor.h
CXLoadedDiagnostic.h
CXSourceLocation.h
CXString.h
CXTranslationUnit.h
CXType.h
Index_Internal.h
)
set(LIBCLANG_INDEX_H Index.h)This list of static archives to an libclang app needs to link against took
some experimentation, apparently we need all them in this approximate order to
link successfully, I have no idea why I just tried stuff until it worked.
set(LIBCLANG_LINK_LIBS
clangAST
clangBasic
clangDriver
clangFrontend
clangIndex
clangLex
clangSema
clangSerialization
clangTooling
clangARCMigrate
LLVMAArch64CodeGen
LLVMAArch64AsmParser
LLVMAArch64Desc
LLVMAArch64Disassembler
LLVMAArch64Info
LLVMAArch64Utils
LLVMAMDGPUCodeGen
LLVMAMDGPUAsmParser
LLVMAMDGPUDesc
LLVMAMDGPUDisassembler
LLVMAMDGPUInfo
LLVMAMDGPUUtils
LLVMARMCodeGen
LLVMARMAsmParser
LLVMARMDesc
LLVMARMDisassembler
LLVMARMInfo
LLVMARMUtils
LLVMBPFCodeGen
LLVMBPFAsmParser
LLVMBPFDesc
LLVMBPFDisassembler
LLVMBPFInfo
LLVMHexagonCodeGen
LLVMHexagonAsmParser
LLVMHexagonDesc
LLVMHexagonDisassembler
LLVMHexagonInfo
LLVMLanaiCodeGen
LLVMLanaiAsmParser
LLVMLanaiDesc
LLVMLanaiDisassembler
LLVMLanaiInfo
LLVMMipsCodeGen
LLVMMipsAsmParser
LLVMMipsDesc
LLVMMipsDisassembler
LLVMMipsInfo
LLVMMSP430CodeGen
LLVMMSP430AsmParser
LLVMMSP430Desc
LLVMMSP430Disassembler
LLVMMSP430Info
LLVMNVPTXCodeGen
LLVMNVPTXDesc
LLVMNVPTXInfo
LLVMPowerPCCodeGen
LLVMPowerPCAsmParser
LLVMPowerPCDesc
LLVMPowerPCDisassembler
LLVMPowerPCInfo
LLVMRISCVCodeGen
LLVMRISCVAsmParser
LLVMRISCVDesc
LLVMRISCVDisassembler
LLVMRISCVInfo
LLVMSparcCodeGen
LLVMSparcAsmParser
LLVMSparcDesc
LLVMSparcDisassembler
LLVMSparcInfo
LLVMSystemZCodeGen
LLVMSystemZAsmParser
LLVMSystemZDesc
LLVMSystemZDisassembler
LLVMSystemZInfo
LLVMWebAssemblyCodeGen
LLVMWebAssemblyAsmParser
LLVMWebAssemblyDesc
LLVMWebAssemblyDisassembler
LLVMWebAssemblyUtils
LLVMWebAssemblyInfo
LLVMX86CodeGen
LLVMX86AsmParser
LLVMX86Desc
LLVMX86Disassembler
LLVMX86Info
LLVMXCoreCodeGen
LLVMXCoreDesc
LLVMXCoreDisassembler
LLVMXCoreInfo
LLVMCore
LLVMSupport
clangFormat
clangToolingInclusions
clangToolingCore
clangFrontend
clangDriver
LLVMOption
clangParse
clangSerialization
clangSema
clangEdit
clangRewrite
clangAnalysis
clangASTMatchers
clangAST
clangLex
clangBasic
LLVMAArch64Desc
LLVMAArch64Info
LLVMAArch64Utils
LLVMMIRParser
LLVMAMDGPUDesc
LLVMAMDGPUInfo
LLVMAMDGPUUtils
LLVMARMDesc
LLVMARMInfo
LLVMARMUtils
LLVMHexagonDesc
LLVMHexagonInfo
LLVMLanaiDesc
LLVMLanaiInfo
LLVMipo
LLVMVectorize
LLVMIRReader
LLVMAsmParser
LLVMInstrumentation
LLVMLinker
LLVMSystemZDesc
LLVMSystemZInfo
LLVMWebAssemblyDesc
LLVMWebAssemblyInfo
LLVMGlobalISel
LLVMAsmPrinter
LLVMDebugInfoDWARF
LLVMSelectionDAG
LLVMCodeGen
LLVMScalarOpts
LLVMAggressiveInstCombine
LLVMInstCombine
LLVMBitWriter
LLVMTransformUtils
LLVMTarget
LLVMAnalysis
LLVMProfileData
LLVMTextAPI
LLVMObject
LLVMBitReader
LLVMCore
LLVMRemarks
LLVMBitstreamReader
LLVMMCParser
LLVMMCDisassembler
LLVMMC
LLVMBinaryFormat
LLVMDebugInfoCodeView
LLVMDebugInfoMSF
LLVMSupport
LLVMCFGuard
LLVMFrontendOpenMP
LLVMDemangle
LLVMAVRCodeGen
LLVMAVRAsmParser
LLVMAVRDisassembler
LLVMAVRDesc
LLVMAVRInfo
LLVMPasses
LLVMCoroutines
LLVMSupport
LLVMObjCARCOpts
)function(get_libclang_sources_and_headers clang_source_path clang_prebuilt_path result_sources result_headers result_required_libs)
list(TRANSFORM LIBCLANG_SOURCE_FILES PREPEND ${clang_source_path}/${LIBCLANG_SOURCE_PATH}/ OUTPUT_VARIABLE RES)
set(${result_sources} ${RES} PARENT_SCOPE)
unset(RES)
list(TRANSFORM LIBCLANG_ADDITIONAL_HEADER_FILES PREPEND ${clang_source_path}/${LIBCLANG_SOURCE_PATH}/ OUTPUT_VARIABLE RES)
list(TRANSFORM LIBCLANG_INDEX_H PREPEND ${clang_source_path}/${LIBCLANG_INCLUDE_PATH}/ OUTPUT_VARIABLE RES1)
list(APPEND RES ${RES1})
set(${result_headers} ${RES} PARENT_SCOPE)
unset(RES)
if(MSVC)
list(TRANSFORM LIBCLANG_LINK_LIBS PREPEND ${clang_prebuilt_path}/lib/ OUTPUT_VARIABLE RES)
list(TRANSFORM RES APPEND .lib OUTPUT_VARIABLE RES)
else()
list(TRANSFORM LIBCLANG_LINK_LIBS PREPEND ${clang_prebuilt_path}/lib/lib OUTPUT_VARIABLE RES)
list(TRANSFORM RES APPEND .a OUTPUT_VARIABLE RES)
endif()
set(${result_required_libs} ${RES} PARENT_SCOPE)
unset(RES)
endfunction()function (gatherArchives all_archives_directory all_archive_names all_archive_paths)
set(ALL_ARCHIVES_DIRECTORY_LOCAL ${CMAKE_CURRENT_BINARY_DIR}/_all_archives)
foreach(archive_path ${ARGN})
get_filename_component(archive_name ${archive_path} NAME)
list(APPEND ARCHIVE_NAMES_LOCAL ${archive_name})
list(APPEND ARCHIVE_PATHS_LOCAL ${ALL_ARCHIVES_DIRECTORY_LOCAL}/${archive_name})
endforeach()
set(${all_archives_directory} ${ALL_ARCHIVES_DIRECTORY_LOCAL} PARENT_SCOPE)
set(${all_archive_names} ${ARCHIVE_NAMES_LOCAL} PARENT_SCOPE)
set(${all_archive_paths} ${ARCHIVE_PATHS_LOCAL} PARENT_SCOPE)
endfunction()CC=@CMAKE_C_COMPILER@
CFLAGS=-I@MAKEFILE_LIBCLANG_INCLUDE@
LIBS=-L@MAKEFILE_LIBCLANG_LIBDIR@ -lclang_bundled -lstdc++ -lm -ldl -lpthread -lz
OBJ=clang_visitor.o
%.o: %.c
$(CC) -c -o $@ $< $(CFLAGS)
clang_visitor: $(OBJ)
$(CC) -o $@ $^ $(CFLAGS) $(LIBS)
.PHONY: clean
clean:
rm *.o clang_visitorCC=@CMAKE_C_COMPILER@
CFLAGS=-I@MAKEFILE_LIBCLANG_INCLUDE@
LIBS=-L@MAKEFILE_LIBCLANG_LIBDIR@ -lclang -lstdc++ -lm -ldl -lpthread -Wl,-rpath=@MAKEFILE_LIBCLANG_LIBDIR@
OBJ=clang_visitor.o
%.o: %.c
$(CC) -c -o $@ $< $(CFLAGS)
clang_visitor: $(OBJ)
$(CC) -o $@ $^ $(CFLAGS) $(LIBS)
.PHONY: clean
clean:
rm *.o clang_visitorCC=@CMAKE_C_COMPILER@
CFLAGS=-I@MAKEFILE_LIBCLANG_INCLUDE@
LIBDIR=@MAKEFILE_LIBCLANG_LIBDIR@
LIBS=-lclang -lz3 -lstdc++ -ldl -lpthread
OBJ=clang_visitor.o
%.o: %.c
$(CC) -c -o $@ $< $(CFLAGS)
clang_visitor: $(OBJ)
$(CC) -o $@ $^ $(CFLAGS) -L$(LIBDIR) $(LIBS); \
install_name_tool -change libz3.dylib $(LIBDIR)/libz3.dylib $@; \
install_name_tool -add_rpath $(LIBDIR) $@;
.PHONY: clean
clean:
rm *.o clang_visitorcmake_minimum_required(VERSION 3.13)
project(clang_visitor)
add_library(LibclangStatic SHARED IMPORTED)
set_property(TARGET LibclangStatic PROPERTY IMPORTED_LOCATION "@CMAKE_MSVC_LIB_DIR@/clang_static_bundled.lib")
set_property(TARGET LibclangStatic PROPERTY IMPORTED_IMPLIB "@CMAKE_MSVC_LIB_DIR@/clang_static_bundled.lib")
include_directories("@CMAKE_MSVC_INCLUDE_DIR@")
add_executable(clang_visitor clang_visitor.c)
target_link_libraries(clang_visitor LibclangStatic Version)
target_compile_definitions(clang_visitor PUBLIC -D_CINDEX_LIB_)
target_link_options(clang_visitor PUBLIC /NODEFAULTLIB:libcmt.lib)
install(TARGETS clang_visitor)To build this project:
> mkdir build
> cd build
> "C:\Program Files\CMake\bin\cmake.exe" -G "Visual Studio 16 2019" .. -DCMAKE_INSTALL_PREFIX=..
> "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Current\Bin\MSBuild.exe" /m -p:Configuration=Release INSTALL.vcxproj
To run:
> cd ..\bin
> clang_visitor.exe
#ifndef __ENUM__
#define __ENUM__
enum Enum
{
RED = 10,
GREEN = 10 << 2,
BLUE = RED + GREEN
};
#endif // __ENUM__#include <clang-c/Index.h>
#include <clang-c/CXString.h>
#include <stdio.h>
#include <stdlib.h>
enum CXChildVisitResult visitor(CXCursor cursor, CXCursor parent, CXClientData data) {
CXSourceLocation location = clang_getCursorLocation( cursor );
if(!clang_Location_isFromMainFile(location))
return CXChildVisit_Continue;
CXString cxspelling = clang_getCursorSpelling(cursor);
const char* spelling = clang_getCString(cxspelling);
CXString cxkind = clang_getCursorKindSpelling(clang_getCursorKind(cursor));
const char* kind = clang_getCString(cxkind);
printf("Cursor spelling, kind: %s, %s\n", spelling, kind);
clang_disposeString(cxspelling);
clang_disposeString(cxkind);
return CXChildVisit_Recurse;
}
int main(int argc, char** argv) {
CXIndex idx = clang_createIndex(1,1);
CXTranslationUnit tu = clang_createTranslationUnitFromSourceFile(idx, "sample.H", 0, 0, 0, 0);
clang_visitChildren(clang_getTranslationUnitCursor(tu), visitor, 0);
return 0;
}When building if you get linker errors that look like:
undefined reference to `std::_Sp_make_shared_tag::_S_eq(std::type_info const&)
that means you have a gcc/g++ version less than 10 and need to upgrade. I ran into this issue with Debian Buster which is still on 8.3.0 and moving to Bullseye worked.
If upgrading isn’t possible the libclang 10 static build will work with older versions.