Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot build @com_google_protobuf//python/dist:binary_wheel #18750

Open
AtoZdevelopment opened this issue Oct 10, 2024 · 2 comments
Open

Cannot build @com_google_protobuf//python/dist:binary_wheel #18750

AtoZdevelopment opened this issue Oct 10, 2024 · 2 comments
Labels

Comments

@AtoZdevelopment
Copy link

What version of protobuf and what language are you using?
Version: 28.2
Language: Bazel

What operating system (Linux, Windows, ...) and version?
Linux, Ubuntu 22.4

What runtime / compiler are you using (e.g., python version or gcc version)
gcc (GCC) 13.2.0
bazel 7.3.2

What did you do?
Steps to reproduce the behavior:

  1. Create folder with the following content:
    MODULE.bazel
bazel_dep(name = "protobuf", version = "28.2", repo_name = "com_google_protobuf")

What did you expect to see
successfull build of the wheel

What did you see instead?

$ bazel build @com_google_protobuf//python/dist:binary_wheel
DEBUG: /home/az/.cache/bazel/_bazel_az/be14e48a27bba7024ecb06f57ad2a794/external/rules_python~/python/private/bzlmod/python.bzl:44:10: WARNING: Ignoring toolchain 'python_3_8' from module 'rules_fuzzing': Toolchain 'python_3_8' from module 'protobuf' already registered Python version 3.8 and has precedence
DEBUG: /home/az/.cache/bazel/_bazel_az/be14e48a27bba7024ecb06f57ad2a794/external/rules_python~/python/private/bzlmod/python.bzl:44:10: WARNING: Ignoring toolchain 'python_3_9' from module 'rules_fuzzing': Toolchain 'python_3_9' from module 'protobuf' already registered Python version 3.9 and has precedence
DEBUG: /home/az/.cache/bazel/_bazel_az/be14e48a27bba7024ecb06f57ad2a794/external/rules_python~/python/private/bzlmod/python.bzl:44:10: WARNING: Ignoring toolchain 'python_3_10' from module 'rules_fuzzing': Toolchain 'python_3_10' from module 'protobuf' already registered Python version 3.10 and has precedence
DEBUG: /home/az/.cache/bazel/_bazel_az/be14e48a27bba7024ecb06f57ad2a794/external/rules_python~/python/private/bzlmod/python.bzl:44:10: WARNING: Ignoring toolchain 'python_3_11' from module 'rules_fuzzing': Toolchain 'python_3_11' from module 'protobuf' already registered Python version 3.11 and has precedence
DEBUG: /home/az/.cache/bazel/_bazel_az/be14e48a27bba7024ecb06f57ad2a794/external/rules_python~/python/private/bzlmod/python.bzl:44:10: WARNING: Ignoring toolchain 'python_3_12' from module 'rules_fuzzing': Toolchain 'python_3_12' from module 'protobuf' already registered Python version 3.12 and has precedence
DEBUG: /home/az/.cache/bazel/_bazel_az/be14e48a27bba7024ecb06f57ad2a794/external/rules_python~/python/private/bzlmod/python.bzl:44:10: WARNING: Ignoring toolchain 'python_3_11' from module 'rules_python': Toolchain 'python_3_11' from module 'protobuf' already registered Python version 3.11 and has precedence
WARNING: Target pattern parsing failed.
ERROR: Skipping '@com_google_protobuf//python/dist:binary_wheel': error loading package '@@protobuf~//python/dist': cannot load '@@rules_python~~python~python_3_12//:version.bzl': no such file
ERROR: error loading package '@@protobuf~//python/dist': cannot load '@@rules_python~~python~python_3_12//:version.bzl': no such file
INFO: Elapsed time: 3.423s
INFO: 0 processes.
ERROR: Build did NOT complete successfully
FAILED: 
    Fetching repository @@bazel_features~; starting

Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs).

Anything else we should know about your project / environment
We ship our own software package including python and python wheels. One of those is the protobuf wheel.
We also depend on the cpp message module.
Building @com_google_protobuf//python/dist:message_mod fails with the same error.
Creating a wheel from the python sources available at top level lack the message_mod.

@AtoZdevelopment AtoZdevelopment added the untriaged auto added to all issues by default when created. label Oct 10, 2024
@acozzette
Copy link
Member

Thank you for the bug report. I tried it out and was able to reproduce the error.

I don't think we will be able to prioritize fixing this, because we mostly intend for that wheel target to be something that we just build ourselves and publish on PyPI. We're able to build it successfully in our own workspace, but I suspect you're running into this issue since you're building it in a separate workspace that doesn't have Python configured in the same way. We're also still in the process of transitioning to bzlmod, so I don't know if our bzlmod support is complete yet.

However, if you would like to fix this yourself, we would be happy to review pull requests with fixes.

@acozzette acozzette removed the untriaged auto added to all issues by default when created. label Oct 10, 2024
@AtoZdevelopment
Copy link
Author

Thank you for your information and feedback!

I got the build working by removing every @system_python call and set SYSTEM_PYTHON_VERSION manually.
This is by no means a patch that can be merged upstream, but it might give additional information if somebody wants to reproduce or dig deeper.

If we would register system_python as toolchain like in protobuf/WORKSPACE this might work as well, unfortunately I did not get that working yet.

index 42175b5ed..6f8088250 100644
--- BUILD.bazel
+++ BUILD.bazel
@@ -387,6 +387,12 @@ alias(
     visibility = ["//visibility:public"],
 )
 
+alias(
+    name = "binary_wheel",
+    actual = "//python/dist:binary_wheel",
+    visibility = ["//visibility:public"],
+)
+
 alias(
     name = "python_srcs",
     actual = "//python:python_srcs",
diff --git python/BUILD.bazel python/BUILD.bazel
index 3e34bbdbc..093fc80f9 100644
--- python/BUILD.bazel
+++ python/BUILD.bazel
@@ -138,8 +138,8 @@ selects.config_setting_group(
 # begin:github_only
 _message_target_compatible_with = {
     "@platforms//os:windows": ["@platforms//:incompatible"],
-    "@system_python//:none": ["@platforms//:incompatible"],
-    "@system_python//:unsupported": ["@platforms//:incompatible"],
+#    "@system_python//:none": ["@platforms//:incompatible"],
+#    "@system_python//:unsupported": ["@platforms//:incompatible"],
     "//conditions:default": [],
 }
 
diff --git python/dist/BUILD.bazel python/dist/BUILD.bazel
index 265d9b885..7caa7b6ed 100644
--- python/dist/BUILD.bazel
+++ python/dist/BUILD.bazel
@@ -11,13 +11,15 @@ load("@pip_deps//:requirements.bzl", "requirement")
 load("@rules_pkg//pkg:mappings.bzl", "pkg_files", "strip_prefix")
 load("@rules_pkg//pkg:tar.bzl", "pkg_tar")
 load("@rules_python//python:packaging.bzl", "py_wheel")
-load("@system_python//:version.bzl", "SYSTEM_PYTHON_VERSION")
+#load("@system_python//:version.bzl", "SYSTEM_PYTHON_VERSION")
 load("//:protobuf_version.bzl", "PROTOBUF_PYTHON_VERSION")
 load(":dist.bzl", "py_dist", "py_dist_module")
 load(":py_proto_library.bzl", "py_proto_library")
 
 licenses(["notice"])
 
+SYSTEM_PYTHON_VERSION = "3"
+
 py_dist_module(
     name = "message_mod",
     extension = "//python:_message_binary",
@@ -272,7 +274,7 @@ pkg_tar(
     package_file_name = "protobuf.tar.gz",
     strip_prefix = ".",
     target_compatible_with = select({
-        "@system_python//:none": ["@platforms//:incompatible"],
+#        "@system_python//:none": ["@platforms//:incompatible"],
         "//conditions:default": [],
     }),
 )
@@ -291,7 +293,7 @@ genrule(
         mv protobuf/dist/*.tar.gz $@
     """,
     target_compatible_with = select({
-        "@system_python//:none": ["@platforms//:incompatible"],
+#        "@system_python//:none": ["@platforms//:incompatible"],
         "//conditions:default": [],
     }),
     tools = [requirement("setuptools")],
@@ -304,6 +306,7 @@ py_wheel(
         "//python:full_api_3.9": "cp39",
         "//conditions:default": "abi3",
     }),
+    visibility = ["//visibility:public"],
     author = "[email protected]",
     author_email = "[email protected]",
     classifiers = [
@@ -350,7 +353,7 @@ py_wheel(
         "src/",
     ],
     target_compatible_with = select({
-        "@system_python//:none": ["@platforms//:incompatible"],
+#        "@system_python//:none": ["@platforms//:incompatible"],
         "//conditions:default": [],
     }),
     version = PROTOBUF_PYTHON_VERSION,
@@ -390,7 +393,7 @@ py_wheel(
         "src/",
     ],
     target_compatible_with = select({
-        "@system_python//:none": ["@platforms//:incompatible"],
+#        "@system_python//:none": ["@platforms//:incompatible"],
         "//conditions:default": [],
     }),
     version = PROTOBUF_PYTHON_VERSION,
@@ -416,7 +419,7 @@ py_wheel(
         "src/",
     ],
     target_compatible_with = select({
-        "@system_python//:none": ["@platforms//:incompatible"],
+#        "@system_python//:none": ["@platforms//:incompatible"],
         "//conditions:default": [],
     }),
     version = PROTOBUF_PYTHON_VERSION,
diff --git python/dist/dist.bzl python/dist/dist.bzl
index 061125ede..74e46b6f7 100644
--- python/dist/dist.bzl
+++ python/dist/dist.bzl
@@ -1,7 +1,8 @@
 """Rules to create python distribution files and properly name them"""
 
 load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo")
-load("@system_python//:version.bzl", "SYSTEM_PYTHON_VERSION")
+#load("@system_python//:version.bzl", "SYSTEM_PYTHON_VERSION")
+SYSTEM_PYTHON_VERSION = "3"
 
 def _get_suffix(limited_api, python_version, cpu):
     """Computes an ABI version tag for an extension module per PEP 3149."""
diff --git python/internal.bzl python/internal.bzl
index 8f99becec..428e1c5cc 100644
--- python/internal.bzl
+++ python/internal.bzl
@@ -127,7 +127,7 @@ def internal_py_test(deps = [], **kwargs):
         imports = ["."],
         deps = deps + ["//python:python_test_lib"],
         target_compatible_with = select({
-            "@system_python//:supported": [],
+#            "@system_python//:supported": [],
             "//conditions:default": ["@platforms//:incompatible"],
         }),
         **kwargs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants