Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: mtree_spec generates incorrect spec file if the root directory has a file and a directory #851

Closed
sin-ack opened this issue May 23, 2024 · 3 comments · Fixed by #852
Labels
bug Something isn't working

Comments

@sin-ack
Copy link
Contributor

sin-ack commented May 23, 2024

What happened?

I'm using a vendored version of py_layer at work (from https://github.com/aspect-build/bazel-examples/blob/fba56194cd925b8d12b0339becbfc10bb9504c26/oci_python_image/py_layer.bzl). py_layer uses mtree_spec and tar from aspect_bazel_lib to create multiple tar files which are then passed to oci_image. As the input, I'm passing a py_binary like this:

py_binary(
    name = "project_bin",
    srcs = ["project/__main__.py"],
    deps = [":project_lib"],
    imports = ["."],
    main = "project/__main__.py",
)

py_oci_image(
    name = "image",
    base = "@distroless_python",
    binary = ":project_bin",
    entrypoint = ["/usr/bin/python3", "/project_bin"],
)

However, this doesn't work properly. The reason is because the mtree spec generated by aspect_bazel_lib looks like this:

project uid=0 gid=0 time=1672560000 mode=0755 type=dir
project_bin uid=0 gid=0 time=1672560000 mode=0755 type=file content=bazel-out/k8-fastbuild/...
project/__main__.py uid=0 gid=0 time=1672560000 mode=0755 type=file content=project/__main__.py

This unfortunately hits some (frankly very bizarre) rules in the mtree file format, specifically this one:

Relative
If the first whitespace-delimited word has no / characters, it is the name of a file in the current directory. Any relative entry that describes a directory changes the current directory.

This means that when bsdtar processes the first line, it "switches directory" to project/, and creates project/project_bin in the tar file rather than project_bin.

Additionally, we can't use /project_bin because mtree treats leading slashes as "special":

Special
Lines beginning with / are special commands that influence the interpretation of later lines.

./project_bin works, but shows up as ./project_bin in tar tf output as well (not sure if that breaks anything). It seems the only proper way to place files at the root directory is to make sure that files at root are always listed before the first directory entry.

Version

Development (host) and target OS/architectures: Gentoo Linux amd64

Output of bazel --version: 7.1.2

Version of the Aspect rules, or other relevant rules from your
WORKSPACE or MODULE.bazel file:

bazel_dep(name = "aspect_bazel_lib", version = "2.7.3")

Language(s) and/or frameworks involved: Python

How to reproduce

Run the following commands:

$ mkdir repro
$ cd repro
$ mkdir foo
$ touch foo/__main__.py
$ touch bar
$ cat > mtree.spec <<EOF
foo uid=0 gid=0 time=1672560000 mode=0755 type=dir
bar uid=0 gid=0 time=1672560000 mode=0644 type=file content=bar
foo/__main__.py uid=0 gid=0 time=1672560000 mode=0644 type=file content=foo/__main__.py
EOF
$ bsdtar -c -f repro.tar @mtree.spec
$ tar -tf repro.tar
foo/
foo/bar
foo/__main__.py

Any other information?

No response

@sin-ack sin-ack added the bug Something isn't working label May 23, 2024
@github-actions github-actions bot added the untriaged Requires traige label May 23, 2024
@gregmagolan gregmagolan removed the untriaged Requires traige label May 23, 2024
@gregmagolan
Copy link
Collaborator

cc @thesayyn

@thesayyn
Copy link
Collaborator

thesayyn commented May 23, 2024

That makes sense, IMHO non-breaking way to fix this is to add trailing slash to the dir entries so they are not interpreted as any special entry. Do you want to make a pr for it?

@sin-ack
Copy link
Contributor Author

sin-ack commented May 23, 2024

Sure!

sin-ack added a commit to sin-ack/bazel-lib that referenced this issue May 23, 2024
bsdtar's mtree format has a quirk wherein entries without "/" in their
first word are treated as "relative" entries, and "relative" directories
will cause tar to "change directory" into the declared directory entry.
If such a directory is followed by a "relative" entry, then the file
will be created within the directory, instead of at top-level as
expected. To mitigate, we append a slash to top-level directory entries.

Fixes bazel-contrib#851.
sin-ack added a commit to sin-ack/bazel-lib that referenced this issue May 27, 2024
bsdtar's mtree format has a quirk wherein entries without "/" in their
first word are treated as "relative" entries, and "relative" directories
will cause tar to "change directory" into the declared directory entry.
If such a directory is followed by a "relative" entry, then the file
will be created within the directory, instead of at top-level as
expected. To mitigate, we append a slash to top-level directory entries.

Fixes bazel-contrib#851.
sin-ack added a commit to sin-ack/bazel-lib that referenced this issue May 27, 2024
bsdtar's mtree format has a quirk wherein entries without "/" in their
first word are treated as "relative" entries, and "relative" directories
will cause tar to "change directory" into the declared directory entry.
If such a directory is followed by a "relative" entry, then the file
will be created within the directory, instead of at top-level as
expected. To mitigate, we append a slash to top-level directory entries.

Fixes bazel-contrib#851.
sin-ack added a commit to sin-ack/bazel-lib that referenced this issue Jun 19, 2024
bsdtar's mtree format has a quirk wherein entries without "/" in their
first word are treated as "relative" entries, and "relative" directories
will cause tar to "change directory" into the declared directory entry.
If such a directory is followed by a "relative" entry, then the file
will be created within the directory, instead of at top-level as
expected. To mitigate, we append a slash to top-level directory entries.

Fixes bazel-contrib#851.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants