cgroups isolation

[dzmitry-lahoda]

so, with docker we have decent isolation, but need copy all - slow.

with nix, it has cgroups, it has partial isolation - but shares binaries - they hashed - so no binary deps hell.

with process compose, i am hacking so that 2 process share files without even visible in yaml about it.

also cannot split 2 processes networks.

if i start thinking about exporting compose file from process compose, cgroups/chroot for some files and some cgroup for ports makes sence.

really, on linux(btw, windows and mac has sandboxing without container copy all too) process compose can be made better than docker and docker compose.

because it may isolate write some reads and ports listen, but still share deps.

[dzmitry-lahoda]

sorry for haiku style of writing, just trying to say there is niche with no tool.

there are containerd/podman/docker/fobarbaz which copy files - slow.

but process compose can make step closer to these (may be using systemd/launchd/ windows whatever or build your own docker in 200 bash lines - see how hacker new about it).

really, i have never seen such product ever.

[dzmitry-lahoda]

will donate 100USD in crypto port cgroups support.

[adrian-gierakowski]

What do you intend to isolate with cgroups? Cgroups are for resource management (cpu, ram etc). For network and disk isolation you need to use namespaces.

Btw for network and file system isolation have a look at https://github.com/nixpak/nixpak

maybe you could use it together with pc and https://github.com/Platonic-Systems/process-compose-flake

[dzmitry-lahoda]

it would be hard just use nixpak and pcf to do isolation.

example, i have process A and B should be in one network. C and D in other. B and D should have public ports available on host.

in order to have A and B in same network, how i can express that? make one process? but than these process are not composed, but one, in pc.

but yeah, isolating one by one if there is nice access to cgroups, like pcf(pc(nixpak(package))) would work.

see example

https://docs.docker.com/compose/networking/#specify-custom-networks

it has A it own network, C its own, but D can access these both.

sure I can use Docker, but than I would need to build.

sure I can build my own part of process manager, but than it will defeat purpose of pc.

[adrian-gierakowski]

How about https://github.com/hercules-ci/arion with podman? It avoids copying stuff around by bing mounting hosts /nix/store into the containers

[dzmitry-lahoda]

we used that. so we used by specifying usual content and entry point. which copied whole closure.

in theory can make entrypoint to call something from mount without closure. may work i guess. it will assemble dummy containers all the time.

so config will be ugly.

not sure if podman is good here, it is alfa something was in arion. we used docker. docker required root install. also not ideal.

intresting idea.

other point, some tool, called zombient we used, depended on process start and required 6 other binaries, so will need to form farm links and yet to avoid copies.

so it is not one binary copy.

so more nice config and no root still could be awesome in pc, and tbh serviced could be better fit as more integrated and no need to install.

[dzmitry-lahoda]

i see some release in june which could be intresting. we started year ago.

[dzmitry-lahoda]

yeah, they added some features hercules-ci/arion#207 . so it may do what i wanted

[dzmitry-lahoda]

https://iximiuz.com/en/posts/you-dont-need-an-image-to-run-a-container/

so in theory pc can just run bundles on linux)))

i am not sure if windows or mac can do.

[dzmitry-lahoda]

https://github.com/phylum-dev/birdcage seems closes thing possible. in YAML of PC add additional metadata. so that some kind of wrapper reads it, and instead of running exe directly wraps than itno Mac/Linux sandbox, and generates new yaml.

so making this part of PC is just simpler

[dzmitry-lahoda]

https://news.ycombinator.com/item?id=37407758

[adrian-gierakowski]

Yes, this is awesome. I saw the announcement in a three on the opencontainers repo earlier today. It does make running local k8s cluster for development and e2e tests much smoother. I’d still use process-compose for day to day dev and integration testing though.

[adrian-gierakowski]

So, if you really need container-style isolation, you should use this with containerd + nerdctl which has docker-compose compatible interface

[adrian-gierakowski]

In the light of the above I’d suggest that resource isolation should be considered out of scope for process-compose and that this discussion should be closed

[dzmitry-lahoda]

imho isolation as security measure for sure is out of scope.

but isolation as way to test that process does not effects/use (until process has some hacks) some random directories and binaries.
and isolation to avoid conflicts of 2 processes using same folder.
or isolation of cpu/mem/disk to observe behavior of process under various limits.
it can and eventually should be.

security isolation is high cost and slow.

but security to test/debug and avoid conflicts is fast and cheap.

#249 (comment)

[dzmitry-lahoda]

I definitely need such things from time to time and use nix for that. But there should be nix free option which works on all platfroms.