Daemonization and/or readiness check

[thenonameguy]

It would be great to have explicit support for daemonization of process-compose.
Since the project already has an HTTP RPC API and a disable TUI flag, the next logical step would be to have a:

docker-compose up -d

equivalent.

For local development this is not a big concern for most, although there are people that prefer not to have a TUI
They just haven't used process-compose's. 😂

In a CI environment the lack of this feature becomes apparent.
Let's say you described your service dependency tree in process-compose and you need the full tree running AND ready to be able to run your integration tests.
It would be insanely convenient to have a simple CI script like:

process-compose up --daemonize-after-service-tree-ready  -p $PORT --timeout 10 # seconds
./my-app-test.sh
process-compose down -p $PORT

You can kind of achieve this today with cachix/devenv#83 (comment) but it's POSIX API specific (no windows).

Thoughts? :) @F1bonacc1

[thenonameguy]

An alternative, potentially more granular implementation is to specify the above with a repeatable --wait-until-ready $PROCESS_NAME cli option, that "blocks" until all the mentioned processes reach their healthy state. Then the ---daemonize-after-ready flag is just a shorthand for specifying all the "top-level" processes in the tree.

[F1bonacc1]

To tell you the truth I have been thinking about this feature for some time now.
And it's always great to see that there is a real need for a specific feature and I don't just add them for the fun of the challenge :)

Personally, most of the time I use tmux, even for local terminals, but I see how it will benefit the project to not have to learn and install an additional tool.

I will close the 2 open issues and have fun with daemonizing it :)

[thenonameguy]

Thanks, that's good to hear :)

One thing I realized after creating the discussion and browsing some Go daemonization repos/blog posts is that it's not as easy as it seems on the surface.
Now that the general sentiment around daemons in the Go community is to 'just not do it', how else could we tackle this problem?

That got me thinking about my previous comment about supporting the -f syntax of docker-compose to deep-merge process yaml description into a single DAG (it's not really a tree with multiple root nodes).

This way the ./my-app-test.sh is just an extra layer on top of the dependency graph, acting as a oneshot service in systemd terms.

Now the only extension we need to solve the up -> wait till top-layer processes exit 0/1 -> down problem is some way of specification in the yaml spec.

process-compose up -f compose.yaml -f compose.ci.yaml --exit-once-complete unit-test --exit-once-complete integration-test --timeout 300 # seconds

The presence of the --exit-once-complete flag could automatically apply TUI=false flag so the logs are visible in CI.
The exit value (after graceful shutdown) can be the maximum exit code along all the processes (this way if any process fails with exit code 1, the surrounding script gets notified).

This should be more extensible than daemonization, as any compose.yaml specific improvement applies to these tasks.

Now the CI case should be covered.

For the local dev, background process-compose users we still need a solution to simulate -d, but we can do that right after exec, which should be in line with existing go daemonization examples and less cumbersome than a delayed daemonization.

[F1bonacc1]

Yeah, after some googling I also saw that it's not a simple go get github.com/make/me/a/daemon...
Regarding your specific use case today this is the default behavior, process-compose should exit once all the procs are done.
So assuming the following DAG:

graph TD;
    TestA-->ProcA;
    TestA-->ProcB;
    TestA-->ProcC;
    TestB-->TestA
    Cleanup-->TestB

Loading

The tests can run when A,B,C are ready and exit on failure if TestA or TestB fails, with this configuration:
https://github.com/F1bonacc1/process-compose#-terminate-process-compose-on-failure

[dzmitry-lahoda]

so is there analog of docker-compose down?

[F1bonacc1]

@dzmitry-lahoda
This is not specifically related to this discussion, but yes, process-compose down will be a part of the next release.
I'm curious, what is the use case for it?

[dzmitry-lahoda]

It is similar to thread initial scenario

process-compose up --daemonize-after-service-tree-ready  -p $PORT --timeout 10 # seconds
./my-app-test.sh
process-compose down -p $PORT

Launch some set of processes, run tests, teardown.

[dzmitry-lahoda]

process compose win here docker because it can do that faster (no image build step)

[F1bonacc1]

It's part of v0.69.0 now