Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bzlmod #238

Draft
wants to merge 28 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
12dfe57
squash
lalten Feb 1, 2024
64ea98a
bzlmoded curl
lalten Apr 2, 2024
b3d2b08
pybind11
lalten Apr 2, 2024
4cc7d12
pre-commit
lalten Apr 2, 2024
0217b58
fix examples WS load
lalten Apr 2, 2024
db3cad8
python version shenanigans
lalten Apr 2, 2024
cec3c76
pre-commit
lalten Apr 2, 2024
e2b926c
POC: extra_targets alias in rules_python to make numpy headers available
lalten May 15, 2024
1e16de8
POC: use whl_filegroup
lalten May 16, 2024
01af1e4
split off dev_pip, fix issue with six not being found due to escaping…
lalten May 21, 2024
9900f35
Merge branch 'main' into bzlmod
lalten Jun 2, 2024
7efb38d
Merge branch 'main' into bzlmod
lalten Jun 29, 2024
7c59bad
upgrade rules_python
lalten Jun 29, 2024
25bbe0e
follow https://rules-python.readthedocs.io/en/latest/toolchains.html#…
lalten Jun 29, 2024
ebdf5d8
examples rules_python version
lalten Jun 29, 2024
7cd2365
rm pip annotations (using whl_filegroup now)
lalten Jun 29, 2024
3a1779e
bazelrc
lalten Jun 29, 2024
c8e9ef4
fix examples?
lalten Jun 29, 2024
7243f76
gitignore
lalten Jun 29, 2024
435e95a
Fix plugin aspect for py version transition
lalten Jun 29, 2024
fa2c658
simplify
lalten Jun 29, 2024
8ae4038
empty commit to retrigger CI on potentially flaky `//third_party/foxg…
lalten Jun 29, 2024
77d2a3c
mark test as flaky
lalten Jun 29, 2024
4d492f3
allow flaky tests...
lalten Jun 29, 2024
b7b5a5d
Merge branch 'main' into bzlmod
lalten Aug 29, 2024
c57a5c9
sync versions
lalten Aug 29, 2024
e662ac5
is_default=True
lalten Aug 29, 2024
2b06741
Add rules_rust
lalten Aug 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions .bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,9 @@
# Mandatory at the moment for Bazel 7.0.0.
build --noincompatible_sandbox_hermetic_tmp

# This is mandatory if your build env doesn't have a proper Python 3 exe
# -- like the CI build image for this repo.
# The build image is using a fake python exe, see
# https://github.com/mvukov/bazel_builder/blob/main/Dockerfile.
# If blzmod is enabled, but not used as in this repo at the moment),
# then bazel for some reason tries to auto-detect a Python exe.
build --noenable_bzlmod
# Don't let local Python site packages leak into the build and cause problems
common --incompatible_strict_action_env
common --action_env=PYTHONNOUSERSITE=1

# Fix the wrong default to generate __init__.py to delimit a Python package.
# This is a mandatory flag.
Expand All @@ -37,6 +33,11 @@ build --noexperimental_check_output_files
# Don't bother building targets which aren't dependencies of the tests.
test --build_tests_only

# Show all the problems in CI
test --keep_going
test --test_output=errors
build --verbose_failures

# To use a clang compiler, invoke Bazel with `--config=clang`.
build:clang --repo_env=CC=clang
build:clang --repo_env=CXX=clang++
Expand Down
4 changes: 1 addition & 3 deletions .github/workflows/ci.bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,7 @@ build:remote --noslim_profile

# Don't spam CI logs.
build --show_progress_rate_limit=60
common --color=yes

# Don't rely on test logs being easily accessible from the test runner,
# though it makes the log noisier.
test --test_output=errors
# Allows tests to run bazelisk-in-bazel, since this is the cache folder used
test --test_env=XDG_CACHE_HOME
24 changes: 20 additions & 4 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,18 +53,26 @@ jobs:

# Run bazel test with gcc and clang in each workspace
strategy:
fail-fast: false
matrix:
folder:
- "."
- "examples"
toolchain:
- "gcc"
- "clang"
external_dependency_system:
- bzlmod
- workspace
include:
- toolchain: "gcc"
config_option: ""
- toolchain: "clang"
config_option: "--config=clang"
- external_dependency_system: bzlmod
bzlmod_flags: --enable_bzlmod
- external_dependency_system: workspace
bzlmod_flags: --noenable_bzlmod

steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
Expand All @@ -89,13 +97,21 @@ jobs:
path: |
~/.cache/bazel
~/.cache/bazel-repo
key: v3-bazel-repo-cache-${{ matrix.toolchain }}-${{ hashFiles('.bazelversion', 'examples/WORKSPACE', 'repositories/**', 'requirements_lock.txt', 'WORKSPACE') }}
restore-keys: v3-bazel-repo-cache-${{ matrix.toolchain }}-
- name: bazel test ${{ matrix.config_option }} //...
key: >-
v3-bazel-repo-cache
-${{ matrix.external_dependency_system }}-${{ matrix.toolchain }}
-${{ hashFiles('.bazelversion', '**/WORKSPACE', '**/MODULE.bazel', 'repositories/**/*.bzl', 'requirements_lock.txt') }}
restore-keys: >-
v3-bazel-repo-cache
-${{ matrix.external_dependency_system }}-${{ matrix.toolchain }}
- name: bazel test //...
env:
# Bazelisk will download bazel to here, ensure it is cached between runs.
XDG_CACHE_HOME: ~/.cache/bazel-repo
BUILDBUDDY_ORG_API_KEY: ${{ secrets.BUILDBUDDY_ORG_API_KEY }}
USER: ${{ needs.configure.outputs.user_name }}
working-directory: ${{ matrix.folder }}
run: ${GITHUB_WORKSPACE}/.github/workflows/test.sh ${{ matrix.config_option }}
run: >
${GITHUB_WORKSPACE}/.github/workflows/test.sh
${{ matrix.config_option }}
${{ matrix.bzlmod_flags }}
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@
bazel-*
__pycache__
user.bazelrc
external/
1 change: 0 additions & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
default_language_version:
python: python3.10
node: 18.16.0

default_stages: [commit]
Expand Down
18 changes: 17 additions & 1 deletion BUILD.bazel
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
load("@rules_python//python:pip.bzl", "compile_pip_requirements")
load("@rules_cc//cc:defs.bzl", "cc_library")
load("@rules_python//python:pip.bzl", "whl_filegroup")
load("@rules_ros2_pythons//3.10:defs.bzl", "compile_pip_requirements")

compile_pip_requirements(
name = "python_requirements",
Expand All @@ -12,3 +14,17 @@ exports_files([
"requirements.txt",
"requirements_lock.txt",
])

whl_filegroup(
name = "numpy_includes",
pattern = "numpy/core/include/numpy",
whl = "@rules_ros2_pip_deps//numpy:whl",
)

cc_library(
name = "rules_ros2_pip_deps_numpy_headers",
hdrs = [":numpy_includes"],
includes = ["numpy_includes/numpy/core/include"],
visibility = ["//visibility:public"],
deps = ["@rules_python//python/cc:current_py_cc_headers"],
)
102 changes: 102 additions & 0 deletions MODULE.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
module(name = "com_github_mvukov_rules_ros2")

bazel_dep(name = "asio", version = "1.28.2")
bazel_dep(name = "bazel_skylib", version = "1.6.1")
bazel_dep(name = "boringssl", version = "0.0.0-20240126-22d349c")
bazel_dep(name = "curl", version = "8.4.0")
bazel_dep(name = "eigen", version = "3.4.0")
bazel_dep(name = "fmt", version = "10.2.1")
bazel_dep(name = "libyaml", version = "0.2.5")
bazel_dep(name = "lz4", version = "1.9.4")
bazel_dep(name = "nlohmann_json", version = "3.11.3")
bazel_dep(name = "platforms", version = "0.0.8")
bazel_dep(name = "pybind11_bazel", version = "2.11.1.bzl.3")
bazel_dep(name = "readerwriterqueue", version = "1.0.6")
bazel_dep(name = "rules_cc", version = "0.0.9")
bazel_dep(name = "rules_foreign_cc", version = "0.10.1")
bazel_dep(name = "rules_python", version = "0.33.2")

bazel_dep(name = "googletest", version = "1.14.0.bcr.1", dev_dependency = True)

bazel_dep(name = "spdlog", version = "1.12.0")
bazel_dep(name = "sqlite3", version = "3.42.0.bcr.1")
bazel_dep(name = "tinyxml2", version = "10.0.0")
bazel_dep(name = "websocketpp", version = "0.8.2")
bazel_dep(name = "yaml-cpp", version = "0.8.0")
bazel_dep(name = "zlib", version = "1.3")
bazel_dep(name = "zstd", version = "1.5.5.bcr.1")

_DEFAULT_PYTHON = "3.10"

python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(python_version = _DEFAULT_PYTHON)
lalten marked this conversation as resolved.
Show resolved Hide resolved
use_repo(python, rules_ros2_pythons = "python_versions")

pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse(
hub_name = "rules_ros2_pip_deps",
python_version = _DEFAULT_PYTHON,
requirements_lock = "//:requirements_lock.txt",
)
use_repo(pip, "rules_ros2_pip_deps")

dev_pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip", dev_dependency = True)
dev_pip.parse(
hub_name = "rules_ros2_resolver_deps",
python_version = _DEFAULT_PYTHON,
requirements_lock = "//repositories/private:resolver_requirements_lock.txt",
)
use_repo(dev_pip, "rules_ros2_resolver_deps")

pybind_configure = use_extension("@pybind11_bazel//:internal_configure.bzl", "internal_configure_extension")
use_repo(pybind_configure, "pybind11")
lalten marked this conversation as resolved.
Show resolved Hide resolved

non_module_deps = use_extension("@com_github_mvukov_rules_ros2//:extensions.bzl", "non_module_deps")
use_repo(
non_module_deps,
"cyclonedds",
"foxglove_bridge",
"iceoryx",
"osrf_pycommon",
"ros2",
"ros2_ament_cmake_ros",
"ros2_ament_index",
"ros2_class_loader",
"ros2_common_interfaces",
"ros2_diagnostics",
"ros2_geometry2",
"ros2_image_common",
"ros2_kdl_parser",
"ros2_keyboard_handler",
"ros2_launch",
"ros2_launch_ros",
"ros2_libstatistics_collector",
"ros2_message_filters",
"ros2_pluginlib",
"ros2_rcl",
"ros2_rcl_interfaces",
"ros2_rcl_logging",
"ros2_rclcpp",
"ros2_rclpy",
"ros2_rcpputils",
"ros2_rcutils",
"ros2_resource_retriever",
"ros2_rmw",
"ros2_rmw_cyclonedds",
"ros2_rmw_dds_common",
"ros2_rmw_implementation",
"ros2_robot_state_publisher",
"ros2_ros_testing",
"ros2_rosbag2",
"ros2_rosidl",
"ros2_rosidl_python",
"ros2_rosidl_runtime_py",
"ros2_rosidl_typesupport",
"ros2_rpyutils",
"ros2_tracing",
"ros2_unique_identifier_msgs",
"ros2_urdfdom",
"ros2_urdfdom_headers",
"ros2_xacro",
"ros2cli",
)
13 changes: 6 additions & 7 deletions WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -10,22 +10,21 @@ load("//repositories:deps.bzl", "ros2_deps")

ros2_deps()

load("@rules_python//python:repositories.bzl", "py_repositories", "python_register_toolchains")
load("@rules_python//python:repositories.bzl", "py_repositories", "python_register_multi_toolchains")

py_repositories()

python_register_toolchains(
name = "rules_ros2_python",
python_version = "3.10",
python_register_multi_toolchains(
name = "rules_ros2_pythons",
default_version = "3.10",
python_versions = ["3.10"],
)

load("@rules_python//python:pip.bzl", "pip_parse")
load("@rules_ros2_python//:defs.bzl", python_interpreter_target = "interpreter")
load("//repositories:pip_annotations.bzl", "PIP_ANNOTATIONS")
load("@rules_ros2_pythons//3.10:defs.bzl", python_interpreter_target = "interpreter")

pip_parse(
name = "rules_ros2_pip_deps",
annotations = PIP_ANNOTATIONS,
python_interpreter_target = python_interpreter_target,
requirements_lock = "@com_github_mvukov_rules_ros2//:requirements_lock.txt",
)
Expand Down
Empty file added WORKSPACE.bzlmod
Empty file.
59 changes: 59 additions & 0 deletions examples/MODULE.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
module(name = "rules_ros2_examples")

bazel_dep(name = "com_github_mvukov_rules_ros2")
local_path_override(
module_name = "com_github_mvukov_rules_ros2",
path = "..",
)

non_module_deps = use_extension("@com_github_mvukov_rules_ros2//:extensions.bzl", "non_module_deps")
use_repo(
non_module_deps,
"cyclonedds",
"foxglove_bridge",
"iceoryx",
"osrf_pycommon",
"ros2_ament_cmake_ros",
"ros2_ament_index",
"ros2_class_loader",
"ros2_common_interfaces",
"ros2_diagnostics",
"ros2_geometry2",
"ros2_image_common",
"ros2_kdl_parser",
"ros2_keyboard_handler",
"ros2_launch",
"ros2_launch_ros",
"ros2_libstatistics_collector",
"ros2_message_filters",
"ros2_pluginlib",
"ros2_rcl",
"ros2_rcl_interfaces",
"ros2_rcl_logging",
"ros2_rclcpp",
"ros2_rclpy",
"ros2_rcpputils",
"ros2_rcutils",
"ros2_resource_retriever",
"ros2_rmw",
"ros2_rmw_cyclonedds",
"ros2_rmw_dds_common",
"ros2_rmw_implementation",
"ros2_robot_state_publisher",
"ros2_ros_testing",
"ros2_rosbag2",
"ros2_rosidl",
"ros2_rosidl_python",
"ros2_rosidl_runtime_py",
"ros2_rosidl_typesupport",
"ros2_rpyutils",
"ros2_tracing",
"ros2_unique_identifier_msgs",
"ros2_urdfdom",
"ros2_urdfdom_headers",
"ros2_xacro",
"ros2cli",
)

bazel_dep(name = "rules_cc", version = "0.0.9")
bazel_dep(name = "rules_python", version = "0.33.2")
13 changes: 6 additions & 7 deletions examples/WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -26,22 +26,21 @@ load("@com_github_mvukov_rules_ros2//repositories:deps.bzl", "ros2_deps")

ros2_deps()

load("@rules_python//python:repositories.bzl", "py_repositories", "python_register_toolchains")
load("@rules_python//python:repositories.bzl", "py_repositories", "python_register_multi_toolchains")

py_repositories()

python_register_toolchains(
name = "rules_ros2_python",
python_version = "3.10",
python_register_multi_toolchains(
name = "rules_ros2_pythons",
default_version = "3.10",
python_versions = ["3.10"],
)

load("@com_github_mvukov_rules_ros2//repositories:pip_annotations.bzl", "PIP_ANNOTATIONS")
load("@rules_python//python:pip.bzl", "pip_parse")
load("@rules_ros2_python//:defs.bzl", python_interpreter_target = "interpreter")
load("@rules_ros2_pythons//3.10:defs.bzl", python_interpreter_target = "interpreter")

pip_parse(
name = "rules_ros2_pip_deps",
annotations = PIP_ANNOTATIONS,
python_interpreter_target = python_interpreter_target,
requirements_lock = "@com_github_mvukov_rules_ros2//:requirements_lock.txt",
)
Expand Down
8 changes: 8 additions & 0 deletions extensions.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
load("@com_github_mvukov_rules_ros2//repositories:repositories.bzl", "ros2_repositories")

def _non_module_deps_impl(mctx):
ros2_repositories()

non_module_deps = module_extension(
implementation = _non_module_deps_impl,
)
1 change: 1 addition & 0 deletions external
3 changes: 0 additions & 3 deletions repositories/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -1,12 +1,9 @@
load("@rules_python//python:defs.bzl", "py_binary")

exports_files([
"deps.bzl",
"logging_macros.bzl",
"generate_rclcpp_interface.py",
"generate_rclcpp_logging_macros.py",
"generate_rcutils_logging_macros.py",
"pip_annotations.bzl",
"rclcpp_interfaces.bzl",
"repositories.bzl",
"ros2_repo_mappings.yaml",
Expand Down
13 changes: 4 additions & 9 deletions repositories/asio.BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,10 @@ load("@rules_cc//cc:defs.bzl", "cc_library")

cc_library(
name = "asio",
srcs = glob(["src/*.cpp"]),
hdrs = glob([
"include/**/*.hpp",
"include/**/*.ipp",
srcs = glob([
"**/*.hpp",
"**/*.ipp",
]),
defines = [
"ASIO_SEPARATE_COMPILATION",
],
includes = ["include"],
includes = ["."],
visibility = ["//visibility:public"],
deps = ["@boringssl//:ssl"],
)
Loading