Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove /etc/passwd and /etc/group parsing on runc run/exec #3999

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

kolyshkin
Copy link
Contributor

@kolyshkin kolyshkin commented Aug 29, 2023

  1. libct: switch to numeric UID/GID/groups

    This addresses the following TODO in the code (added back in 2015
    by commit 845fc65):

    // TODO: fix libcontainer's API to better support uid/gid in a typesafe way.

    Historically, libcontainer internally uses strings for user, group, and
    additional (aka supplementary) groups.

    Yet, runc receives those credentials as part of runtime-spec's process,
    which uses integers for all of them (see 1, 2).

    What happens next is:

    1. runc start/run/exec converts those credentials to strings (a User
      string containing "UID:GID", and a []string for additional GIDs) and
      passes those onto runc init.
    2. runc init converts them back to int, in the most complicated way
      possible (parsing container's /etc/passwd and /etc/group).

    All this conversion and, especially, parsing is totally unnecessary,
    but is performed on every container exec (and start).

    The only benefit of all this is, a libcontainer user could use user and
    group names instead of numeric IDs (but runc itself is not using this
    feature, and we don't know if there are any other users of this).

    Let's remove this back and forth translation, hopefully increasing
    runc exec performance.

    The only remaining need to parse /etc/passwd is to set HOME environment
    variable for a specified UID, in case $HOME is not explicitly set in
    process.Env. This can now be done right in prepareEnv, which simplifies
    the code flow a lot. Alas, we can not use standard os/user.LookupId, as
    it could cache host's /etc/passwd or the current user (even with the
    osusergo tag).

    PS Note that the structures being changed (initConfig and Process) are
    never saved to disk as JSON by runc, so there is no compatibility issue
    for runc users.

    Still, this is a breaking change in libcontainer, but we never promised
    that libcontainer API will be stable (and there's a special package
    that can handle it -- github.com/moby/sys/user). Reflect this in
    CHANGELOG.

    For 3998.

  2. libct: earlier Rootless vs AdditionalGroups check

    Since the UID/GID/AdditonalGroups fields are now numeric,
    we can address the following TODO item in the code (added
    by commit d2f4969 back in 2016):

    TODO: We currently can't do
    this check earlier, but if libcontainer.Process.User was typesafe
    this might work.

    Move the check to much earlier phase, when we're preparing
    to start a process in a container.

  3. runc list: use standard os/user

    Switch from github.com/moby/sys/user to Go stdlib os/user
    (which has both libc-backed and pure Go implementations).

  4. libct/user: remove

    This package is marked as deprecated in v1.2.0, and have no internal
    users now, so let's remove it (for v1.3.0).

Fixes: #3998

@kolyshkin kolyshkin force-pushed the uidgid branch 2 times, most recently from 28aadc5 to 4866cd2 Compare August 29, 2023 03:35
@kolyshkin kolyshkin changed the title Uidgid Remove /etc/passwd and /etc/group parsing on runc run/exec Aug 29, 2023
@thaJeztah
Copy link
Member

@kolyshkin kolyshkin force-pushed the uidgid branch 2 times, most recently from 95e76a7 to ab0c62a Compare September 28, 2023 23:54
@kolyshkin kolyshkin marked this pull request as ready for review September 28, 2023 23:58
@kolyshkin kolyshkin marked this pull request as draft June 13, 2024 03:57
@kolyshkin
Copy link
Contributor Author

This is still a draft pending #4325 merge -- once that one is merged, I will rework setUser here.

@kolyshkin kolyshkin force-pushed the uidgid branch 2 times, most recently from bb70a3b to e87bc0e Compare January 9, 2025 23:44
@kolyshkin kolyshkin added this to the 1.3.0 milestone Jan 9, 2025
@kolyshkin kolyshkin force-pushed the uidgid branch 2 times, most recently from 214d77a to 348c250 Compare January 10, 2025 01:01
@kolyshkin kolyshkin marked this pull request as ready for review January 10, 2025 01:29
@kolyshkin
Copy link
Contributor Author

kolyshkin commented Jan 10, 2025

This seems to be ready now. @AkihiroSuda @cyphar @lifubang @rata PTAL. Overall I think it's a big step forward:

  • addressing a couple of (rather glaring) TODOs from 2015/2016;
  • straightening libct API;
  • removing code that shouldn't be there and does nothing;
  • all without breaking runc's backward compatibility (hopefully).

Also, I know that the second commit is a bit too big for review. I tried to split it into two but that seems unfeasible, as the two features (changing user string to uid:gid and moving the setting of HOME to prepareEnv) are interdependent.

Historically, when HOME is not explicitly set in process.Env,
and UID to run as doesn't have a corresponding entry in container's
/etc/passwd, runc sets HOME=/ as a fallback.

Add the corresponding check, for the sake of backward compatibility
preservation.

Signed-off-by: Kir Kolyshkin <[email protected]>
This addresses the following TODO in the code (added back in 2015
by commit 845fc65):

> // TODO: fix libcontainer's API to better support uid/gid in a typesafe way.

Historically, libcontainer internally uses strings for user, group, and
additional (aka supplementary) groups.

Yet, runc receives those credentials as part of runtime-spec's process,
which uses integers for all of them (see [1], [2]).

What happens next is:

1. runc start/run/exec converts those credentials to strings (a User
   string containing "UID:GID", and a []string for additional GIDs) and
   passes those onto runc init.
2. runc init converts them back to int, in the most complicated way
   possible (parsing container's /etc/passwd and /etc/group).

All this conversion and, especially, parsing is totally unnecessary,
but is performed on every container exec (and start).

The only benefit of all this is, a libcontainer user could use user and
group names instead of numeric IDs (but runc itself is not using this
feature, and we don't know if there are any other users of this).

Let's remove this back and forth translation, hopefully increasing
runc exec performance.

The only remaining need to parse /etc/passwd is to set HOME environment
variable for a specified UID, in case $HOME is not explicitly set in
process.Env. This can now be done right in prepareEnv, which simplifies
the code flow a lot. Alas, we can not use standard os/user.LookupId, as
it could cache host's /etc/passwd or the current user (even with the
osusergo tag).

PS Note that the structures being changed (initConfig and Process) are
never saved to disk as JSON by runc, so there is no compatibility issue
for runc users.

Still, this is a breaking change in libcontainer, but we never promised
that libcontainer API will be stable (and there's a special package
that can handle it -- github.com/moby/sys/user). Reflect this in
CHANGELOG.

For 3998.

[1]: https://github.com/opencontainers/runtime-spec/blob/v1.0.2/config.md#posix-platform-user
[2]: https://github.com/opencontainers/runtime-spec/blob/v1.0.2/specs-go/config.go#L86

Signed-off-by: Kir Kolyshkin <[email protected]>
Since the UID/GID/AdditonalGroups fields are now numeric,
we can address the following TODO item in the code (added
by commit d2f4969 back in 2016):

> TODO: We currently can't do
> this check earlier, but if libcontainer.Process.User was typesafe
> this might work.

Move the check to much earlier phase, when we're preparing
to start a process in a container.

Signed-off-by: Kir Kolyshkin <[email protected]>
Switch from github.com/moby/sys/user to Go stdlib os/user
(which has both libc-backed and pure Go implementations).

Signed-off-by: Kir Kolyshkin <[email protected]>
This package is marked as deprecated in v1.2.0, and have no internal
users, so let's remove it (for v1.3.0).

Signed-off-by: Kir Kolyshkin <[email protected]>
@kolyshkin kolyshkin requested review from rata and thaJeztah and removed request for rata January 10, 2025 20:26
@kolyshkin kolyshkin requested review from lifubang and removed request for thaJeztah January 10, 2025 20:26
@kolyshkin kolyshkin requested review from rata and thaJeztah January 15, 2025 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RFC: remove user/group name support from libcontainer
4 participants