[Bug]: xz not found with new statically linked bsdtar

[qtica]

What happened?

When upgrading to bazel-lib 2.7.0, the new statically linked bsdtar when used by rules_distroless cacerts results in the error:

tar: Error opening archive: Can't initialize filter; unable to run program "xz -d -qq"

I would expect there to be no regression in behavior when upgrading to the new static solution. However, if it is expected for the system to provide xz, then I can update my images for RBE to contain xz.

Version

Development (host) and target OS/architectures:

Output of bazel --version: 7.1.1

Version of the Aspect rules, or other relevant rules from your
WORKSPACE or MODULE.bazel file: 2.7.0

Language(s) and/or frameworks involved: rules_distroless v0.3.1

How to reproduce

gh repo clone GoogleContainerTools/rules_distroless

cd rules_distroless

cat <<EOF >> .bazelrc.user
build:docker-sandbox --experimental_docker_image=docker.io/library/debian:12.2-slim
build:docker-sandbox --spawn_strategy=docker
build:docker-sandbox --strategy=Javac=docker
build:docker-sandbox --strategy=Closure=docker
build:docker-sandbox --strategy=Genrule=docker
build:docker-sandbox --define=EXECUTOR=remote
build:docker-sandbox --experimental_docker_verbose
build:docker-sandbox --experimental_enable_docker_sandbox
EOF

# MODULE.bazel (update aspect_bazel_lib to 2.7.0)
sed -i 's/"aspect_bazel_lib", version = "[0-9.]\+"/"aspect_bazel_lib", version = "2.7.0"/g' MODULE.bazel

# On version 2.7.0 this fails (full error below)
bazel build //examples/cacerts:cacerts --config=docker-sandbox

# MODULE.bazel (revert back to 2.6.0)
sed -i 's/"aspect_bazel_lib", version = "[0-9.]\+"/"aspect_bazel_lib", version = "2.6.0"/g' MODULE.bazel

# On version 2.6.0 this passes
bazel build //examples/cacerts:cacerts --config=docker-sandbox

Any other information?

Full Error:

INFO: Analyzed target //examples/cacerts:cacerts (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
INFO: Docker sandboxing is supported
ERROR: ~/projects/github.com/GoogleContainerTools/rules_distroless/examples/cacerts/BUILD.bazel:4:8: Action examples/cacerts/cacerts.crt failed: (Exit 1): cacerts.sh failed: error executing command (from target //examples/cacerts:cacerts) distroless/private/cacerts.sh external/aspect_bazel_lib~2.7.0~toolchains~bsd_tar_linux_amd64/tar external/example-bullseye-ca-certificates/data.tar.xz ... (remaining 2 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
tar: Error opening archive: Can't initialize filter; unable to run program "xz -d -qq"
Target //examples/cacerts:cacerts failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: ~/projects/github.com/GoogleContainerTools/rules_distroless/examples/cacerts/BUILD.bazel:4:8 CaCertsTarGz examples/cacerts/cacerts.tar.gz failed: (Exit 1): cacerts.sh failed: error executing command (from target //examples/cacerts:cacerts) distroless/private/cacerts.sh external/aspect_bazel_lib~2.7.0~toolchains~bsd_tar_linux_amd64/tar external/example-bullseye-ca-certificates/data.tar.xz ... (remaining 2 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
INFO: Elapsed time: 0.488s, Critical Path: 0.40s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully

[alexeagle]

I agree that's not a desired behavior change. @dzbarsky is the expert on that pre-built bsd tar, I'm not sure if the answer is that xz ought to be statically linked, or if we're meant to call tar with some flags to specify where it can locate an xz binary

[DavidZbarsky-at]

My guess is we are not defining HAVE_LIBLZMA when building so we don't take this codepath https://github.com/libarchive/libarchive/blob/1a2fd29f6de7decd097dc4e30629ff96a068f72e/libarchive/archive_read_support_filter_xz.c#L124

Then we end up trying to shell out to system xz instead of using the statically linked lib.

I'll try to send up a patch in the next few days, @alexeagle might make sense to add a test to bazel-lib and/or the bsdtar released repo, wdyt?

[dzbarsky]

@qtica I couldn't repro with your instruction on darwin, but I was able to repro .tar.xz files not working, and defining HAVE_LIBLZMA in compilation fixed that. So hopefully it will fix your issue as well. We saw one more report of .tar.gz not working correctly so I'm hoping to get a repro for that and bundle the fixes, since adding the module to BCR is a bit tedious. Thanks for your patience and stay tuned!

[alexeagle]

I believe it's fixed by #819

[qtica]

I've confirmed from my side that the latest fix does resolve my issue. Thank you for the quick replies and fix, I just got back from vacation and it's nice to see it's already released in a patch!

I couldn't repro with your instruction on darwin,

I should've specified that the above instructions were on an amd64 linux machine. 🙇 But I'm glad you were able to find another way to reproduce it.