v3.0 Roadmap

[waruqi]

If there are proposed changes that you would like to see introduced in xmake v3.0, you can submit them here.

It will be adopted if the vote result >= 80%.

we can use set_policy("compatibility.version", "3.0") or xmake f --policies=compatibility.version:3.0 to attempt v3.0

Roadmap (Adopted)

• Use MD/MDd runtimes for msvc by default
  • Poll: Use MD/MDd runtimes for msvc by default #5526
• Use ninja generator for cmake package by default
  • Poll: Use ninja generator for cmake package by default #5545
• Simplifying add_requires with configs
  • Simplifying boolean value configs #5727 (comment)

Roadmap (Draft)

• Remove support for built-in variables in script scope, support it only in the description
  • Poll: Whether or not to remove support for built-in variables in script scope? #5529
• Rename buildir to builddir
  • Poll: Rename buildir to builddir #5533
  • Use ninja generator for cmake package by default #5596
• Improve custom toolchain to support unknown compiler/linker.
  • Improve custom toolchain to support unknown compiler/linker. #5554
• Support project namespace
  • Handle target name clashes in subprojects (Support namespace) #3380
• Autobuild when calling xmake run
  • Poll: Enable auto build when calling `xmake run` by default #5624
• ...

[24bit-xjkp]

Support dynamical script load, so we can distribute some xmake scripts by packages. This would be useful when sharing some scripts between projects.

[waruqi]

Support dynamical script load, so we can distribute some xmake scripts by packages. This would be useful when sharing some scripts between projects.

This key point is difficult to be implemented because the package is not yet installed when it is first compiled. We can currently only distribute packages rules.

[SirLynix]

A better way for xmake packages to pull off info from their build xmake.lua, to avoid having to duplicate defines and such

[heheda123123]

I think it is very important to improve the documentation, now the documentation of a few more serious problems
1 Even if I select Chinese in xmake.io, the search still comes up with English results
2 Some methods are not found in the documentation

[KkemChen]

支持对apt，pkg-config等系统库指定使用动态库或静态库

[jingkaimori]

elements of target

Re-consider relationship of rules, toolchains, config variables, values and policies.

Table below shows which entity can hold which element:

entity type	rule	toolchain	config variable	value	policies
option			v
target	v	v	v	v	v
(global space)	v	v	v		v
package
xpack

Option

should option be a type of task, or just a object like value? should option checks itself?

[waruqi]

I think it is very important to improve the documentation, now the documentation of a few more serious problems 1 Even if I select Chinese in xmake.io, the search still comes up with English results 2 Some methods are not found in the documentation

new document, https://docs.xmake.io/en-us/index.html

But I don't have a lot of time alone to update and perfect it.

[24bit-xjkp]

Maybe we can use document generators like LDoc and embed the document into annotations.

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Supports specifying the use of dynamic libraries or static libraries for system libraries such as apt and pkg-config.

[waruqi]

Remove support for built-in variables in script scope, support it only in the description scope, Poll: #5529

[qudix]

A way to reuse rules in a project that uses xmake, but in a package definition without duplicating work.
https://github.com/shad0wshayd3-FO4/CommonLibF4/blob/master/CommonLibF4/xmake.lua#L133

If such a thing requires some modifications on the lib side, that's fine.

[WubiCookie]

Rename buildir to builddir

[waruqi]

Rename buildir to builddir

Poll: #5533

[A2va]

For my project xmake-rs, I wanted xmake to send the resolved list of a target's links to the rust part. This way the user wouldn't need to link anything in the build.rs.
To do this, I had to copy a large part of the xmake source code, my idea would be to add a target:orderlinks() function which would return a resolved list of links on a binary target.
I thought about using target:linkcmd(), but the flags are different for different compilers and you'd have to parse them.

[star-hengxing]

Use ninja generator for cmake package by default.

[waruqi]

Use ninja generator for cmake package by default.

Poll: #5545

[jube]

Add a way to really clean everything. Sometimes, I need to delete .xmake directories (and/or build directory) in order to fix a build that should work. A nice command like xmake mrproper would be nice to delete everything, as if it was a fresh installation of xmake (like on a CI).

[waruqi]

Add a way to really clean everything. Sometimes, I need to delete .xmake directories (and/or build directory) in order to fix a build that should work. A nice command like xmake mrproper would be nice to delete everything, as if it was a fresh installation of xmake (like on a CI).

xmake f -c will ignore all configuration cache.

[SirLynix]

A way to change config without having to respecify all config, maybe separate some configs such as mode from other configs. It's quite painful to work with a lot of config because of that.

[KkemChen]

希望option域里面 能够支持 add_rules 接口

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

It is hoped that the add_rules interface can be supported in the option domain.

[xq114]

希望定义option时可以直接指定default值，而不用这样再包一层 https://github.com/xmake-io/xmake-repo/blob/dev/packages/a/angle/port/6288/xmake.lua

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

I hope that when defining options, I can directly specify the default value without having to wrap it up again https://github.com/xmake-io/xmake-repo/blob/dev/packages/a/angle/port/6288/xmake.lua

[waruqi]

希望定义option时可以直接指定default值，而不用这样再包一层 https://github.com/xmake-io/xmake-repo/blob/dev/packages/a/angle/port/6288/xmake.lua

原本就可以直接指定，这里的问题是你用了 is_plat ，想要在 option 里用，只能在脚本域。这个是本身机制决定的，没法在描述域实现

[heheda123123]

跑测试的时候，是否有必要把所有的config也测试到呢？
比如下面的pr，其实是测试不到新添加的config的
xmake-io/xmake-repo#5115

或者说写包描述的时候可以给config添加一个字段指定是否对这个config进行自动测试

[waruqi]

跑测试的时候，是否有必要把所有的config也测试到呢？ 比如下面的pr，其实是测试不到新添加的config的 xmake-io/xmake-repo#5115

或者说写包描述的时候可以给config添加一个字段指定是否对这个config进行自动测试

这个不太现实，新增一个config ，排列组合下，ci 的测试 job 是数量级暴增的，一个 config 如果不是 boolean 值，是 string values 的，组合就更多， 还可能引入各种 deps 编译，没这么多资源，不仅耗时，耗ci资源，跑出问题，也没这么多人力修。

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

When running tests, is it necessary to test all configs? For example, in the PR below, the newly added config cannot be tested xmake-io/xmake-repo#5115

Or when writing a package description, you can add a field to the config to specify whether to automatically test this config.

This is not realistic. If you add a new config, the number of ci test jobs will increase by orders of magnitude under the permutation and combination. Without so many resources, it will not only be time-consuming, but also consume ci resources. If problems occur, there will not be so many manpower to repair them.

[Arteiimis]

#5609

[heheda123123]

#5609

支持，现在的musl.cc工具链只能从linux上跨平台/架构编译。这个工具链能大大扩展交叉编译的能力

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

#5609

Supported, the current musl.cc toolchain can only be compiled cross-platform/architecture from Linux. This tool chain can greatly expand cross-compilation capabilities

[waruqi]

We can use set_policy("compatibility.version", "3.0") or xmake f --policies=compatibility.version:3.0 to attempt v3.0

f9196df

[waruqi]

New Poll: Autobuild when calling xmake run
#5624

[waruqi]

Now we can use set_policy("compatibility.version", "3.0") to switch vs runtime to MD/Md

#5626

[SnowinterCat]

我需要一个接口来遍历我的项目依赖的三方库的列表，我需要将三方库的二进制打包到我的安装包中。
我现在的做法是在脚本域中使用以下代码来获取三方库的二进制的：

        import("core.project.project")
        cprint("${yellow}Export Shared libs")
        for pkg_name, info in pairs(project:required_packages()) do
            local libfiles = info:get("libfiles")
        end

但是这只能获取到三方库的库文件，对于一些拥有插件或者工具的三方库，例如gstreamer、ffmpeg、qt，我需要用到他们的工具和插件，并将其打包到安装包中。目前我没有办法做到直接从xmake的脚本域中找到它们
能否提供一个函数例如project:all_packages(),target:all_packages()来获取到他们的manifest信息？

又或者为package的包管理中添加一个on_export_runtime()，可以用于一键导出三方库的运行时？

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

I need an interface to traverse the list of third-party libraries that my project depends on, and I need to package the binaries of the third-party libraries into my installation package.
My current approach is to use the following code in the script domain to obtain the binary of the third-party library:

        import("core.project.project")
        cprint("${yellow}Export Shared libs")
        for pkg_name, info in pairs(project:required_packages()) do
            local libfiles = info:get("libfiles")
        end

But this can only obtain the library files of third-party libraries. For some third-party libraries with plug-ins or tools, such as gstreamer, ffmpeg, and qt, I need to use their tools and plug-ins and package them into the installation package. Currently I have no way to find them directly from xmake's script domain
Can you provide a function such as project:all_packages(), target:all_packages() to obtain their manifest information?

Or add an on_export_runtime() to the package management of the package, which can be used to export the runtime of third-party libraries with one click?

[waruqi]

我需要一个接口来遍历我的项目依赖的三方库的列表，我需要将三方库的二进制打包到我的安装包中。 我现在的做法是在脚本域中使用以下代码来获取三方库的二进制的：

        import("core.project.project")
        cprint("${yellow}Export Shared libs")
        for pkg_name, info in pairs(project:required_packages()) do
            local libfiles = info:get("libfiles")
        end

但是这只能获取到三方库的库文件，对于一些拥有插件或者工具的三方库，例如gstreamer、ffmpeg、qt，我需要用到他们的工具和插件，并将其打包到安装包中。目前我没有办法做到直接从xmake的脚本域中找到它们 能否提供一个函数例如project:all_packages(),target:all_packages()来获取到他们的manifest信息？

又或者为package的包管理中添加一个on_export_runtime()，可以用于一键导出三方库的运行时？

这种问题，没必要在这里问，可以到 discussion 里提，target:pkgs() ，pkg:installdir() 原本就能取到。

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

I need an interface to traverse the list of third-party libraries that my project depends on, and I need to package the binaries of the third-party libraries into my installation package. My current approach is to use the following code in the script domain to obtain the binary of the third-party library:

import("core.project.project")
cprint("${yellow}Export Shared libs")
for pkg_name, info in pairs(project:required_packages()) do
local libfiles = info:get("libfiles")
end

But this can only obtain the library files of third-party libraries. For some third-party libraries with plug-ins or tools, such as gstreamer, ffmpeg, and qt, I need to use their tools and plug-ins and package them into the installation package. Currently, I have no way to find them directly from the script domain of xmake. Can you provide a function such as project:all_packages(), target:all_packages() to get their manifest information?

Or add an on_export_runtime() to the package management of the package, which can be used to export the runtime of third-party libraries with one click?

There is no need to ask this kind of question here. You can mention it in the discussion. Target:pkgs() and pkg:installdir() can be obtained originally.

[Wzshun]

A way to change config without having to respecify all config, maybe separate some configs such as mode from other configs. It's quite painful to work with a lot of config because of that.

I also want it, something like a file xmake.lua.user, and xmake build will check it if change, auto config with it.

[jingkaimori]

c51和sdcc工具链的默认plat不要设为当前系统，建议为裸机（没有系统的单片机，自立实现）添加名为bare或者standalone的目标平台plat属性值。c51工具链的默认架构arch建议也弄成mcs51，不然下面的日志太难绷：

> xmake config --clean
checking for platform ... windows
checking for architecture ... x64
checking for Microsoft Visual Studio (x64) version ... 2022
checking for Microsoft C/C++ Compiler (x64) version ... 19.40.33811
> xmake build --rebuild
[ 50%]: cache compiling.release src\main.c
[ 75%]: linking.release hello
[ 75%]: generating.release build\windows\x64\release\hello.hex
[100%]: build ok, spent 0.125s

这个无解，每个 target 可以绑定多个不同 toolchain 的，而全局平台 toolchain 只有一个，跟他们独立。

那xmake-repo里的包，如何适配c51或者sdcc工具链，同时不影响默认工具链的编译？

[waruqi]

c51和sdcc工具链的默认plat不要设为当前系统，建议为裸机（没有系统的单片机，自立实现）添加名为bare或者standalone的目标平台plat属性值。c51工具链的默认架构arch建议也弄成mcs51，不然下面的日志太难绷：

> xmake config --clean
checking for platform ... windows
checking for architecture ... x64
checking for Microsoft Visual Studio (x64) version ... 2022
checking for Microsoft C/C++ Compiler (x64) version ... 19.40.33811
> xmake build --rebuild
[ 50%]: cache compiling.release src\main.c
[ 75%]: linking.release hello
[ 75%]: generating.release build\windows\x64\release\hello.hex
[100%]: build ok, spent 0.125s

这个无解，每个 target 可以绑定多个不同 toolchain 的，而全局平台 toolchain 只有一个，跟他们独立。

那xmake-repo里的包，如何适配c51或者sdcc工具链，同时不影响默认工具链的编译？

add_requires("xxx", {configs = {toolchains = ""}}) 也是可以设置 toolchain 的

[SirLynix]

How would you feel about having

add_requires("xx", { configs = { foo = "bar" } })

automatically add

add_requireconfs("**.xx", { configs = { foo = "bar" }})

?

Also

add_requires("libsdl 2.30")

would add

add_requireconfs("**.libsdl", { version = "2.30" }})

This way, package config and versions would automatically replicate to their dependencies as well, because currently:

add_requires("libsdl 2.28", "libsdl_ttf")

will make xmake install this:

$ xmake f --check
checking for Microsoft Visual Studio (x64) version ... 2022
note: install or modify (m) these packages (pass -y to skip confirm)?
in xmake-repo:
  -> libsdl 2.28.5 [runtimes:"MD"]
  -> libsdl#1 2.30.8 [runtimes:"MD", from:libsdl_ttf]
  -> libsdl_ttf 2.22.0 [runtimes:"MD"]
please input: y (y/n/m)

same goes for

add_requires("libsdl", { configs = {sdlmain = false}})
add_requires("libsdl_ttf")

=>

in xmake-repo:
  -> libsdl 2.30.8 [runtimes:"MD", sdlmain:n]
  -> libsdl#1 2.30.8 [runtimes:"MD", from:libsdl_ttf]
  -> libsdl_ttf 2.22.0 [runtimes:"MD"]
please input: y (y/n/m)

it's been a long time I've been thinking about making a feature request for a policy for this, and I think it would be a great default to have that in xmake 3.0.

what do you think?

(also thinking about it, package.librarydeps.strict_compatibility would be great to enable by default in that case, to avoid subtles bugs when you change your mind about a package config also used as a dependency)

[waruqi]

How would you feel about having add_requires("xx", { configs = { foo = "bar" } }) automatically add add_requireconfs("**.xx", { configs = { foo = "bar" }})? Also add_requires("libsdl 2.30") => add_requireconfs("**.libsdl", { version = "2.30" }}).

This way, package config and versions would automatically replicate to their dependencies as well, because currently:

add_requires("libsdl 2.28", "libsdl_ttf")

will make xmake install this:

$ xmake f --check
checking for Microsoft Visual Studio (x64) version ... 2022
note: install or modify (m) these packages (pass -y to skip confirm)?
in xmake-repo:
  -> libsdl 2.28.5 [runtimes:"MD"]
  -> libsdl#1 2.30.8 [runtimes:"MD", from:libsdl_ttf]
  -> libsdl_ttf 2.22.0 [runtimes:"MD"]
please input: y (y/n/m)

same goes for

add_requires("libsdl", { configs = {sdlmain = false}})
add_requires("libsdl_ttf")

=>

in xmake-repo:
  -> libsdl 2.30.8 [runtimes:"MD", sdlmain:n]
  -> libsdl#1 2.30.8 [runtimes:"MD", from:libsdl_ttf]
  -> libsdl_ttf 2.22.0 [runtimes:"MD"]
please input: y (y/n/m)

it's been a long time I've been thinking about making a feature request for a policy for this, and I think it would be a great default to have that in xmake 3.0.

what do you think?

(also thinking about it, package.librarydeps.strict_compatibility would be great to enable by default in that case, to avoid subtles bugs when you change your mind about a package config also used as a dependency)

The add_requires at the user's top level are completely independent. They should not affect the dependency configurations of other packages, as this can lead to many hidden problems.

[SirLynix]

The add_requires at the user's top level are completely independent. They should not affect the dependency configurations of other packages, as this can lead to many hidden problems.

But in the examples I'm showing this is already causing many issues, if you decide to use libsdl as a shared library, libsdl_ttf will use it as a static library causing bugs (libsdl uses global variables which would not be the same between libsdl and libsdl_ttf in that case).

Another example would be openssl, if you use package locking for example and you're locking your openssl package version, it won't lock it as a dependency (and openssl is used by many packages).
Your app cannot have two differents versions of openssl and it will error out on init.

Having that behavior as an opt-out seems safer to me

[waruqi]

The add_requires at the user's top level are completely independent. They should not affect the dependency configurations of other packages, as this can lead to many hidden problems.

But in the examples I'm showing this is already causing many issues, if you decide to use libsdl as a shared library, libsdl_ttf will use it as a static library causing bugs (libsdl uses global variables which would not be the same between libsdl and libsdl_ttf in that case).

so we should use add_requireconfs

Another example would be openssl, if you use package locking for example and you're locking your openssl package version, it won't lock it as a dependency (and openssl is used by many packages).

This is just another issue about requires-lock. We can improve requires-lock instead of add_requires.

Your app cannot have two differents versions of openssl and it will error out on init.

However, two completely independent targets can be bound to different versions of OpenSSL. If their dependency behaviors are globally modified through add_requires, these unrelated targets will all be affected.

Having that behavior as an opt-out seems safer to me

In the future, I will try to improve and solve this problem, but not by modifying the default behavior of add_requires. Modifying it would lead to even more problems than now.

[SirLynix]

I understand even though I don't agree on the current behavior being safer than what I suggest (especially because it causes subtle bugs one can fix only with advanced xmake usage and knowing how libs depends on each other, my students fall in this trap every year).

(About package lock, the issue arise simply when you set a package version through add_requires, at best it duplicates symbols and at worst it causes weird bugs in your app)

How about having this behavior as an opt-in policy? Because even though we can already do this using add_requireconfs, it means duplicating every add_requires with a add_requireconfs which is redundant and more error-prone.

[waruqi]

I understand even though I don't agree on the current behavior being safer than what I suggest (especially because it causes subtle bugs one can fix only with advanced xmake usage and knowing how libs depends on each other, my students fall in this trap every year).

(About package lock, the issue arise simply when you set a package version through add_requires, at best it duplicates symbols and at worst it causes weird bugs in your app)

How about having this behavior as an opt-in policy? Because even though we can already do this using add_requireconfs, it means duplicating every add_requires with a add_requireconfs which is redundant and more error-prone.

I haven't figured out how to improve and solve it yet. You can open an issue first.

[Bluelari]

A way to change config without having to respecify all config, maybe separate some configs such as mode from other configs. It's quite painful to work with a lot of config because of that.

这个+1，甚至希望xmake config的默认行为就是只修改而不要清除之前的config，类似于git config。并且如果真的需要覆盖已有config的话也可以先跑xmake config --clean再修改

[for2years]

Assuming target A depends on target B, the build.fence policy will cause the object files of target A to wait until target B is fully linked before compiling, resulting in no-full-usage of the CPU.

On the other hand, build.target_across_parallel leads to even lower CPU utilization. Will xmake 3.0.0 consider optimizing CPU utilization for codegen/proto targets? We have a log of codegen / proto tasks in our repo, so we're quite interested in this feature.