Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop containers on interrupts #26

Merged
merged 2 commits into from
Apr 17, 2024

Conversation

eszpotanski
Copy link
Contributor

Currently, when CTRL-C is used, Bazel sends SIGKILL to subprocesses, which only kills script process, but not Docker container.

This PR fixes this behavior, by enabling graceful-termination and handling SIGTERM signal.

Copy link
Collaborator

@oharboe oharboe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rework the code a bit to leave fewer questions and surprises

docker_shell.sh Outdated Show resolved Hide resolved
# Stop or remove container
docker container rm -f "bazel-orfs-$uid" || true
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rework so that the handler is installed only when the container is running?


#!/bin/bash

# Function to handle SIGTERM signal
cleanup() {
    echo "Stopping container..."
    # Command to stop the container gracefully
    docker stop <container_name_or_id>
    exit 0
}

# Main
echo "Starting container..."
# Command to start your container, for example:
# docker run -d --name <container_name> <image_name>
docker run -d --name my_container my_image

# Wait until the container is either running or stopped
while true; do
    container_status=$(docker inspect -f '{{.State.Status}}' my_container)
    if [[ "$container_status" == "running" || "$container_status" == "exited" ]]; then
        break
    fi
    sleep 1
done

# Registering the cleanup function to handle SIGTERM
trap cleanup SIGTERM

# Wait for the script to be terminated
echo "Script is running. Press Ctrl+C to stop the container gracefully."
while true; do
    sleep 1
done

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, SIGTERM can be received after docker run and before trap, then signal won't be handled, but container will be at least created.

I recreated this approach and tested it on BoomTile_synth, where a lot of containers are created, and usually at least one created container remained, but all running containers were stopped and removed.

Current solution stops and removes all containers created during bazel build.

If you prefer to add handler only when container is running and possibly not remove all created containers, I will commit these changes.

@oharboe oharboe merged commit f9d6204 into The-OpenROAD-Project:main Apr 17, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants