[question] Why is this project on hold until after the 2.0 release?

[hwhsu1231]

Recently, I found that Conan decided to deprecate "cmake-conan" in Conan 2.X. For example, in this article of the blog:

The biggest drawback is you need to call conan install before cmake ... This sounds innocent at first but if you rely on third party tools that support cmake you no longer have the cmake-wrapper to call conan for you. The way CMake designed the toolchain support is that it needs to be preset at configuration before the project is declared.

Conan will no longer get its settings from CMake, rather Conan will convert its settings to CMake which you will need to pass explicitly.

Therefore, I want to confirm that:

The main reason is all about the "Chicken-And-Egg" problem which @memsharded mentioned in this issue that I posted before: #410 (comment)

In any case the conan_toolchain.cmake is mostly there to try go guarantee that your CMake configuration follows the Conan settings. As the goal of cmake-conan is the opposite, to obtain the Conan settings from the current CMake configuration, that renders the conan_toolchain.cmake mostly useles, because it should largely contain what is somewhat already defined in the current CMake run.

Am I correct?

[hwhsu1231]

According to my observation, there are some other problems that cause cmake-conan "imcompatiable" with Conan 2.0. For example, conan_cmake_autodetect() "cannot" parse the correct arguments which will be passed to conan_cmake_install().

• Conan 2.0 doesn't accept values from conan_cmake_autodetect() #421

However, the biggest problem for now is the "Chicken-And-Egg (CAE)" problem, isn't it? If so, does that mean Conan Official will put more efforts on supporting and maintaining cmake-conan if someone found or proposed a method or usage that can solve this CAE problem?

After all, it seems that those other problems are not so hard to fix (just need to take some time) compared with the CAE problem.

[memsharded]

The main problem is:

• Users heavily pushed for having a "transparent" CMake integration that doesn't require to modify CMakeLists.txt at all
• That means using find_package() for dependencies and a toolchain.cmake file for configuration
• If Conan is generating the conan_toolchain.cmake file when it is called, and the toolchain file needs to be passed to CMake at invocation time only, can't be done later, then chicken-and-egg, and basically nothing can be done, besides calling conan install and then cmake, in this order
• The only alternative is assume that cmake-conan will work without the toolchain information at all, but that is a very fragile assumption that can bring problems, at the very least a bad user experience.

[hwhsu1231]

@memsharded Thanks for your reply!

And let me check whether I fully understand what you listed:

• For the 1st one, it means that the usage should be as SIMPLE as possible and had better become NON-INTRUSIVE integration with CMake.

• For the 2nd and 4th one, it means that the usage should be worked when we specify CMakeDeps and CMakeToolchain generators in our recipe SIMULTANEOUSLY. And we need to:

  • find_package the xxx-config.cmake (generaeted by CMakeDeps).
  • include the conan_toolchain.cmake (generated by CMakeToolchain) BEFORE the invocation of project() command.

• For the 3rd one, it is exactly what I mentioned above, the CAE problem.

Am I correct? And if someone can propose a METHOD that can properly tackle all these problems, will Conan Official RECONSIDER the cmake-conan?

[memsharded]

Yes, sure, let us know your ideas about this. Just in case, have a look to the repo and previous issues, for example we have considered using the initial cache conan-io/conan#12034, and it didn't move forward.

[hwhsu1231]

@memsharded

In short, my idea is to "rewrap the APIs of cmake-conan into a toolchain file", conan-setup.cmake. For usage, users just need to do the following two things:

• Prepare a conan recipe (conanfile.txt or conanfile.py) alongside with CMakeLists.txt in the root directory.
• Specify cache variable CMAKE_PROJECT_INCLUDE_BEFORE with conan-setup.cmake when running cmake command.

And then, everything will be done automatically.

I've already created a prototype. And I'm still writing the documentation and doing some tests with different compilers and cmake-level generators. So it might take a while. In terms of the usage, do you think it is simple enough? (Suppose that it satisfies all the other conditions)

[memsharded]

Not fully clear yet, so the conan-setup.cmake file will be the one calling conan install? That will result in a conan_toolchain.file. How is this toolchain file passed to cmake, via CMAKE_TOOLCHAIN_FILE variable?

Does this work for CMAKE_PROJECT_INCLUDE_BEFORE? My understanding is that this inclusion happens after the processing of the toolchain file happens, which might be a limitation?

[hwhsu1231]

I will explain the detailed mechanism when proposing the whole project. (please wait for a while)

I guarantee that it can work successfully. The following is the output log of using MSVC compilers:

Click to expand

F:\Git-Repo\cmake-project-template-conan>vcvarsall x64
**********************************************************************
** Visual Studio 2019 Developer Command Prompt v16.11.19
** Copyright (c) 2021 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'

F:\Git-Repo\cmake-project-template-conan>cmake --preset win32-msvc-x64-ninja-multi
Preset CMake variables:

  CMAKE_CONAN_ERROR_QUIET:BOOL="TRUE"
  CMAKE_CONAN_OUTPUT_QUIET:BOOL="TRUE"
  CMAKE_CONFIGURATION_TYPES="Debug;Release"
  CMAKE_CXX_COMPILER="cl.exe"
  CMAKE_C_COMPILER="cl.exe"
  CMAKE_INSTALL_PREFIX:PATH="F:/Git-Repo/cmake-project-template-conan/install/win32-msvc-x64-ninja-multi"
  CMAKE_PROJECT_INCLUDE_BEFORE="F:/Git-Repo/cmake-project-template-conan/cmake/conan-setup.cmake"

-- Using Conan toolchain: F:/Git-Repo/cmake-project-template-conan/build/win32-msvc-x64-ninja-multi/conan-dependencies/conan_toolchain.cmake
-- The C compiler identification is MSVC 19.29.30146.0
-- The CXX compiler identification is MSVC 19.29.30146.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
========== cmake-project-template-conan ==========
-- Conan: Component target declared 'fmt::fmt'
-- Conan: Target declared 'asio::asio'
-- Conan: Target declared 'CLI11::CLI11'
-- Conan: Component target declared 'Eigen3::Eigen'
-- Conan: Component target declared 'libpqxx::pqxx'
-- Conan: Component target declared 'libpq::pgcommon'
-- Conan: Component target declared 'libpq::pgport'
-- Conan: Component target declared 'libpq::pq'
-- Conan: Target declared 'PostgreSQL::PostgreSQL'
-- Conan: Component target declared 'CURL::libcurl'
-- Conan: Component target declared 'OpenSSL::Crypto'
-- Conan: Component target declared 'OpenSSL::SSL'
-- Conan: Target declared 'openssl::openssl'
-- Conan: Including build module from 'D:/.conan/short/d461c0/1/lib/cmake/conan-official-openssl-variables.cmake'
-- Conan: Target declared 'ZLIB::ZLIB'
-- Configuring done
-- Generating done
-- Build files have been written to: F:/Git-Repo/cmake-project-template-conan/build/win32-msvc-x64-ninja-multi

As we can see in the above log message, there are -- Using Conan toolchain: and -- Conan: Target dechlared messages. That is, it indeed includes conan_toolchain.cmake before the project() command and find_packages xxx-config.cmake.

For now, I just want to confirm whether the usage method of my idea is simple enough?

For usage, users just need to do the following two things:

• Prepare a conan recipe (conanfile.txt or conanfile.py) alongside with CMakeLists.txt in the root directory.
• Specify cache variable CMAKE_PROJECT_INCLUDE_BEFORE with conan-setup.cmake when running cmake command.

[memsharded]

Yes, it is looking good for me. Lets see what @czoido and @jcar87 think too.

Transferring this issue to cmake-conan repo, this is something to implement there, I don't think this require changes in the Conan client only in the cmake-conan integration

[Talkless]

That means using find_package() for dependencies and a toolchain.cmake file for configuration

If this gonna be the ONLY way in 2.0 to consume Conan packages, I see it a rather show stopper.

For non-opensource project at work I don't care about "transparency", and since my usages is mainly via QtCreator IDE, keeping convenience of use (that QtCreator + cmake-conan provides) has rather high priority.

For example, if I use cmake-conan, I can:

1. Open CMakeLists.txt with QtCreator, as any other "Project" in any other IDE.
2. Switch to whatever Qt Kit (Desktop, Android,..) I need.
3. Choose Release or Debug or Whatever configuration I need.
4. Just invoke "Build" and conan install is invoked automatically by cmake-conan.

Now, if on Conan 2.0 toolchain generator will be the only way, I will have to make helper scripts like conna_install_desktop_release.sh, conan_install_android_debug.sh, ..., and run them before launching QtCreator and changing Kit and Debug/Release configuration? Will have to hard-code QtCreators "shadow build" automatically-created directories which change depending on used Qt Kit / configuration so that conan install will place required files where needed?

Though maybe I am missing something and over-reacting?

I believe there was some work in QtCreator, Qt itself to make it work better with Conan. @memsharded are you aware of any of that?

[SuTanTank]

In my practice, I only use conan-cmake with CMakeDeps generator to generate xxx-config.cmake files and then add the output directory to CMAKE_PREFIX_PATH. The build configurations inside the toolchain is good, but not that useful for an already-exist CMake project - many of them have their own cmake options to handle build configs on different platforms and even some already has their own toolchains.

Besides, almost all the modern C++ IDEs like CLion, VS 2022, Android Studio, have built-in CMake support, which makes it very inconvenient to manually call conan install with these IDEs cmake workflow.

Another inconvenience is submodule. When a project is consumed as a submodule and add_subdirectory(). conan-cmake works normally and the parent project doesn't need to care about calling conan install as long as conan is installed in the environment, but now the inner project maintainer needs to tell the user: don't use submodule, or call conan install and do a long list of things to adapt to conan and even start to learn how conan works. In the worst case, one needs to introduce conan to all the dev teams in the company and make them use it - it's really difficult.

[Nekto89]

I don't really understand how I will be able to use conan without being able to call it multiple times during CMake generation. In my workflow input for conan is generated dynamically based on what libraries from project have been enabled (there are more than 600 targets). Also some options might be changed dynamically based on CMake variables. All this logic is hidden under many layers of CMake. And it becomes even more complex with more added layers on top of this project that also allow to choose what part should be enabled.

[hwhsu1231]

I will explain the detailed mechanism when proposing the whole project. (please wait for a while)

I've published the whole project. Please take a look:

• Issue: CMake toolchain for integration with Conan using cmake-conan, cmake-conan-setup. #449
• Project: cmake-conan-setup

[parkertomatoes]

Gosh, I wish Conan considered workplace politics when making these design decisions. I don't care about "transparency", I can edit CMakeLists.txt all I want. Getting access to update a build to add some arbitrary shell command because a random dependency demands it?

Imagine spending all your energy and reputation selling a bunch of curmugeony gray beards who have been dumping library sources in VS projects for years because it "just works", to use cmake + conan. "It will make maintenance easier". Then imagine going to each team a few months later, one by one, and telling them that not only there's a breaking change in your newfangled tool, but they need to touch their fragile spit-and-chicken-wire build system setups because of it.

[hwhsu1231]

@parkertomatoes

Execuse me. So what aspects or mechanism of Conan are you talking about?

[jusito]

He stated that cmake-conan is a very important feature for convincing others to integrate conan.
And depending on different build system integrations you would need to add extra handling for a shell invocation conan install ... in between, maybe that is way harder compared to modifying cmake scripts.

[hwhsu1231]

He stated that cmake-conan is a very important feature for convincing others to integrate conan.
And depending on different build system integrations you would need to add extra handling for a shell invocation conan install ... in between, maybe that is way harder compared to modifying cmake scripts.

Agree 100%.

[hwhsu1231]

@SuTanTank

The build configurations inside the toolchain is good, but not that useful for an already-exist CMake project - many of them have their own cmake options to handle build configs on different platforms and even some already has their own toolchains.

Then, what do you think about the integration of Vcpkg with CMake? I'm not sure whether the situation you mentioned will happen or not, but at least this idea is the best for now as far as I know. And I think this is one of reasons why Vcpkg is more widespread than Conan. The usage is SIMPLE enough. (BTW, also NONINTRUSIVE)

[hwhsu1231]

Yes, the usage for pure third-party consuming might be simpler in vcpkg, and that is great for many happy vcpkg users, and we are happy about that. Conan is optimized for package creation, binary management and a huge flexibility. There have to be compromises sometimes and trade-offs, and we are willing to keep improving, learning and trying to provide the simplest usage while keeping all Conan power.

So please stop using that as an argument, and lets keep the conversation technical. The team is extremely busy with the continuous super fast grow of Conan, and there are other higher priorities than this integration at the moment, so you need to be patience too.

@memsharded

No offense. I'm not saying that Conan is worse than Vcpkg. In fact in my opinion, Conan is POTENTIALLY MORE POWERFULL than Vcpkg if all the features it proposed can be fullfiled and more 3rd party libraries can be supported.

• Continuous Integration/Delivery
• Binary Management
• Package Creation
• Flexibility
• ...etc

This is why I tend to put more efforts on Conan. I'm just saying that in terms of Integration with CMake, Vcpkg is doing better than Conan for now. Actually, my cmake-conan-setup project is inspired by Vcpkg at the beginning. (microsoft/vcpkg#24956)

[memsharded]

Ok, thanks very much for the feedback @hwhsu1231, sure no offense, it is just that we are right now absolutely swamped, and it is just too difficult to keep up with everything, and it can be super stressing for us too. We understand there are a few issues that can be generating frustration to users and we are trying our best to address them, but it will take some time, there is simply too much on the backlog, and the migration to 2.0 is a critical priority because the whole situation is very confusing for users at the moment.

We will certainly keep prioritizing the integration with CMake, so far the priority was to remove the intrusive integration that previous approaches did, and we did achieve a fully transparent integration via CMakeDeps and CMakeToolchain (transparent in the sense of not requiring modifications to the CMakeLists.txt, that was the most requested thing by users by far), but of course we will also continue working to try to provide something that also takes care of the caller side. It will just require some time, thanks very much for your patience and understanding.

[SuTanTank]

@SuTanTank

The build configurations inside the toolchain is good, but not that useful for an already-exist CMake project - many of them have their own cmake options to handle build configs on different platforms and even some already has their own toolchains.

Then, what do you think about the integration of Vcpkg with CMake? I'm not sure whether the situation you mentioned will happen or not, but at least this idea is the best for now as far as I know. And I think this is one of reasons why Vcpkg is more widespread than Conan. The usage is SIMPLE enough. (BTW, also NONINTRUSIVE)

I only tried vcpkg in 2019 for a month and I am not sure if it has self-host package server like conan-server and Artifactory, which is the very reason to use conan, because CMake itself does not have this function. Besides, I have spent so much time on conan and don't want to switch to another tool. I have integrated almost all the 3rd-party and 1st-party dependencies in my team with conan, but I'm still selling conan to downstream teams inside the company, who have been using pure CMake and git submudule for years. In my design, downstream can use my library as either a conan binary by conan-camke or add the source code to the project by add_subdirectory(), with a CMake option to control. (Yes, the starting point is always CMake)

vcpkg does have self-hosted package server registries, should I give a try again?

[hwhsu1231]

@SuTanTank

vcpkg does have self-hosted package server registries, should I give a try again?

I don't know because I haven't tried it yet. What I'm saying is that "Vcpkg already provides a toolchain file for integration with CMake". And there is a concept called Manifest Mode, in which we just needs to do the following two things:

• Prepare a Manifest File (vcpkg.json) alongside with the root CMakeListst.txt.
• Specify a cache variable CMAKE_TOOLCHAIN_FILE with its toolchain file vcpkg.cmake.

And then, all the installation of dependencies will be done automatically when we configure our CMake project. As we can see, we don't have to manually run vcpkg install command in Manifest Mode. If you check the README.md of my cmake-conan-setup, you will find that its usage is similar to the Manifest Mode of Vcpkg.

• Prepare a Recipe File (conanfile.py or conanfile.txt) alongside with the root CMakeLists.txt.
• Specify a cache variable CMAKE_PROJECT_INCLUDE_BEFORE with the toolchain file conan-setup.cmake.

Most importantly, I think the integration with CMake should make users manually operate Conan Configurations as less as possible so that users can focus on CMake operations. Ideally, they just need to worry about what libraries their projects depend on and write them down in the Recipe File.

[karl-works]

Like to add another perspective :

I'm using https://github.com/ObKo/stm32-cmake to build SMT32 code using CMake.
By creating some relevant packages i have been able to package our STM32 environment in a very nice compact but highly invasive way.

include(${CMAKE_BINARY_DIR}/conan_paths.cmake)

find_package(GCC-ARM-NONE-EABI REQUIRED)
find_package(STM32-CMAKE REQUIRED)
find_package(STM32-CUBE-G0 REQUIRED)

set(STM32_TARGET_TRIPLET arm-none-eabi)
set(STM32_CUBE_G0_PATH ${CONAN_STM32-CUBE-G0_ROOT})
set(STM32_TOOLCHAIN_PATH ${CONAN_GCC-ARM-NONE-EABI_ROOT}/bin)
set(CMAKE_TOOLCHAIN_FILE ${CONAN_STM32-CMAKE_ROOT}/stm32_gcc.cmake)

I don't see how i can get CMakeToolchain to do what I need and I do not expect Conan to cater to my (Probably doing it wrong) workflow. All I ask is that we do not remove the current cmake_find_package, cmake_paths generators.

[jusito]

@marcus-free you have two questions I guess?

1. How to get the package folder, e.g. CONAN_STM32-CUBE-G0_ROOT:
   It is possible with CMakeDeps generator. E.g. for ZLIB it creates ZLIB-release-x86_64-data.cmake which is loaded after find_package(ZLIB REQUIRED PATHS build) which also sets ${zlib_PACKAGE_FOLDER_RELEASE} to ~/.conan/data/zlib/1.2.13/_/_/package/.../. Its just not nice that you can find only ZLIB but the other variables starting with zlib.
2. You want to chain Toolchain files, thats also possible, if you have your own toolchain file just call the others from it.

Edit:
If there is more of it you need to know, Iam not sure if this would go out of topic, so maybe create a separate issue for it?

[karl-works]

@jusito
Thank you for you patience in responding to my comment.

After you questions I have been iterating on my use case I have realized several things.

• Some very smart people have decided only keep CMakeToolchain and CMakeDeps and they are the domain experts.
• CMakeDeps + DCMAKE_BUILD_TYPE="RELEASE" -DCMAKE_PREFIX_PATH="$PWD" and changes in the CMakeList.txt makes it possible leave the old generators but still keeping my hacky way of doing things.

stm32-cmake uses CONAN_STM32-CUBE-G0 to define a set of CMake Interface Libraries, I'm unsure if it's a good idea to precompile them as this is an MCU use case and we need to give the compiler as much freedom as possible to optimize. (may be wrong about this)

find_package(GCC-ARM-NONE-EABI CONFIG REQUIRED)
find_package(STM32-CMAKE CONFIG REQUIRED)
find_package(STM32-CUBE-G0 CONFIG REQUIRED)

set(STM32_TARGET_TRIPLET arm-none-eabi)
set(STM32_CUBE_G0_PATH ${stm32-cube-g0_BUILD_DIRS_${CMAKE_BUILD_TYPE}})
set(STM32_TOOLCHAIN_PATH ${gcc-arm-none-eabi_BUILD_DIRS_${CMAKE_BUILD_TYPE}}/bin)
set(CMAKE_TOOLCHAIN_FILE ${stm32-cmake_BUILD_DIRS_${CMAKE_BUILD_TYPE}}/stm32_gcc.cmake)

Please ignore my "perspective" I was doing it wrong.

[prince-chrismc]

Please ignore my "perspective" I was doing it wrong.

The glory about CMake is there's more ways to do the same thing than they are atoms in the universe.

We have tried to use the "modern" way that Kitware and the ecosystem have proscribed. In 2020 that was targets and toolchains.

So you aren't wrong just different and thanks Okay ❤️ I am glad to hear you are having some success with the new generators

[karl-works]

Sorry for more of topic but the design Conan 2.0 is a beautiful thing!

f = os.path.join(self.package_folder, "stm32_gcc.cmake")
self.conf_info.define("tools.cmake.cmaketoolchain:user_toolchain", [f])

and

cmake = CMakeDeps(self)
cmake.build_context_activated = ["gcc-arm-none-eabi"]
cmake.generate()

tc = CMakeToolchain(self,"Ninja")
tc.cache_variables["STM32_TARGET_TRIPLET"] = "arm-none-eabi"
tc.cache_variables["STM32_CUBE_G0_PATH"] = self.dependencies.host["stm32-cube-g0"].package_folder
tc.cache_variables["STM32_TOOLCHAIN_PATH"] = os.path.join(self.dependencies.build["gcc-arm-none-eabi"].package_folder,"bin")
tc.generate()

Gives me a CMakeLists.txt that have no idea that Conan exists.

And the conanfile.py if short and sweet!

It's a beautiful thing!

Thanks for your hard work!

[hwhsu1231]

@prince-chrismc @memsharded

Congratulations!!! Just now, Conan 2.0 is officially released. Will the Conan team start to re-evaluate this project and keep maintaining it?

I really think that Conan should provide at least 1 integration in which CMake is the "starting point". I suggest that Conan take Vcpkg's Manifest Mode for reference. Just like I mentioned in this comment: #447 (comment)

[prince-chrismc]

Yes!! This is on the roadmap 🚀 Just be patient 🙏 we haven't forgotten

[MrSparc]

Until now I've been using the cmake_conan wrapper and it allowed me to use CMake support in Visual Studio Code, being able to choose via the VS footbar the CMake build variant (debug, release, etc) and the kit (x86_64_linux, x86_64_windows, etc) .
This made cmake auto-magically configure conan with the right arch and toolchain to use and build binaries:

include(conan.cmake)
conan_cmake_run(CONANFILE conanfile.txt
                BASIC_SETUP
                BUILD missing
                SETTINGS arch=${CMAKE_SYSTEM_PROCESSOR} os=${CMAKE_SYSTEM_NAME})

Since Conan 2.x cmake_conan is unsupported, I tried to see which approach to use, and from what I see in the examples, it forces me to manually invoke conan to configure the toolchain and then run cmake, breaking the CMake IDE integrations .
It would be good to reevaluate this situation and I hope it is possible to have an approach where conan allows running according to the CMake definitions.

[stefansli]

The only thing that's missing is a proper vscode extension that supports the new workflow (generate everything once with conan install then use CMake Presets).

[Talkless]

The only thing that's missing is a proper vscode extension...

An QtCreator extension and X IDE extension and..?

This is huge regression, it's hard to fathom how this was "allowed".

[memsharded]

As @prince-chrismc said #447 (comment), please be patient, this is in our mind, but Conan 2.0 is a massive effort, there are tons of things to do.

[Talkless]

@memsharded Sorry for overreacting. I guess we're just afraid of "losing" amazing tool.

Congrats with 2.0 release and we are eagerly looking forward for any updates!

[memsharded]

Yeah, no prob, we are aware this is a requested and appreciated utility. We have not forgotten, and we will soon be thinking and discussing what can be done to move this forward in a way that makes the most sense for everyone

[hwhsu1231]

@memsharded @prince-chrismc @czoido

If possible, hope Conan team review this PR as soon as possible:

• feat: Detect MSVC settings in Conan 2.0 way #454

If there exist some modification required of this PR, please let me know.

sincerely.

[memsharded]

A proposal for 2.0 based on dependency providers and providing transparent (no need to modify CMakeLists.txt) is in https://github.com/conan-io/cmake-conan/tree/develop2, testing and feedback welcome.

This is no longer on hold, lets close this ticket, and looking forward to new tickets reporting on the new proposal.