Skip to content

Commit

Permalink
Updates to run command (#151)
Browse files Browse the repository at this point in the history 
* test: fix duplicate apps deletion during cleanup
* feat: raise error if no value is provided for a non-boolean option
* feat: add option to delete specific workload to delete command
* feat: add options to use different limit on number of entries and loopback window to logs command
* feat: add option to display logs for specific replica to logs command
* feat: add option to stop specific replica to ps:stop command
* feat: improve run command
* feat: improve handling of commands that accept extra options
* feat: run release script in context of run command
* fix: use exact same image as original workload by default
* fix: wait for runner workload to be updated before starting job
* chore: add note regarding stty size
* test: update specs
* fix: add delay before starting job if image changed
* More log drain variants
* Add log_method option, cleanup
* Fix Kernel.system orphan process handling
  • Loading branch information
@rafaelgomesxyz
rafaelgomesxyz authored May 10, 2024
1 parent d757dbc commit abbb3b6
Show file tree
Hide file tree
Showing 41 changed files with 1,302 additions and 1,163 deletions.
14 changes: 5 additions & 9 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,7 @@ aliases:

# Control Plane offers the ability to use multiple locations.
# default_location is used for commands that require a location
# including `ps`, `run`, `run:detached`, `apply-template`.
# including `ps`, `run`, `apply-template`.
# This can be overridden with option --location=<location> and
# CPLN_LOCATION environment variable.
default_location: aws-us-east-2
Expand Down Expand Up @@ -215,15 +215,15 @@ aliases:

# Workloads that are for the application itself and are using application Docker images.
# These are updated with the new image when running the `deploy-image` command,
# and are also used by the `info`, `ps:`, and `run:cleanup` commands in order to get all of the defined workloads.
# and are also used by the `info` and `ps:` commands in order to get all of the defined workloads.
# On the other hand, if you have a workload for Redis, that would NOT use the application Docker image
# and not be listed here.
app_workloads:
- rails
- sidekiq

# Additional "service type" workloads, using non-application Docker images.
# These are only used by the `info`, `ps:` and `run:cleanup` commands in order to get all of the defined workloads.
# These are only used by the `info` and `ps:` commands in order to get all of the defined workloads.
additional_workloads:
- redis
- postgres
Expand All @@ -233,7 +233,7 @@ aliases:
maintenance_workload: maintenance

# Fixes the remote terminal size to match the local terminal size
# when running the commands `cpl run` or `cpl run:detached`.
# when running `cpl run`.
fix_terminal_size: true

# Apps with a deployed image created before this amount of days will be listed for deletion
Expand All @@ -248,10 +248,6 @@ aliases:
# when running the command `cpl cleanup-images` (`image_retention_max_qty` takes precedence).
image_retention_days: 5

# Run workloads created before this amount of days will be listed for deletion
# when running the command `cpl run:cleanup`.
stale_run_workload_created_days: 2

apps:
my-app-staging:
# Use the values from the common section above.
Expand Down Expand Up @@ -336,7 +332,7 @@ cpl build-image -a tutorial-app
# Run database migrations (or other release tasks) with the latest image,
# while the app is still running on the previous image.
# This is analogous to the release phase.
cpl run:detached rails db:migrate -a tutorial-app --image latest
cpl run -a tutorial-app --image latest -- rails db:migrate

# Pomote the latest image to the app.
cpl deploy-image -a tutorial-app
Expand Down
106 changes: 53 additions & 53 deletions docs/commands.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -105,17 +105,22 @@ cpl copy-image-from-upstream -a $APP_NAME --upstream-token $UPSTREAM_TOKEN --ima

### `delete`

- Deletes the whole app (GVC with all workloads, all volumesets and all images)
- Deletes the whole app (GVC with all workloads, all volumesets and all images) or a specific workload
- Will ask for explicit user confirmation

```sh
# Deletes the whole app (GVC with all workloads, all volumesets and all images).
cpl delete -a $APP_NAME

# Deletes a specific workload.
cpl delete -a $APP_NAME -w $WORKLOAD_NAME
```

### `deploy-image`

- Deploys the latest image to app workloads
- Optionally runs a release script before deploying if specified through `release_script` in the `.controlplane/controlplane.yml` file and `--run-release-phase` is provided
- The release script is run in the context of `cpl run` with the latest image
- The deploy will fail if the release script exits with a non-zero code or doesn't exist

```sh
Expand Down Expand Up @@ -177,13 +182,23 @@ cpl latest-image -a $APP_NAME
### `logs`
- Light wrapper to display tailed raw logs for app/workload syntax
- Defaults to showing the last 200 entries from the past 1 hour before tailing
```sh
# Displays logs for the default workload (`one_off_workload`).
cpl logs -a $APP_NAME
# Displays logs for a specific workload.
cpl logs -a $APP_NAME -w $WORKLOAD_NAME
# Displays logs for a specific replica of a workload.
cpl logs -a $APP_NAME -w $WORKLOAD_NAME -r $REPLICA_NAME
# Uses a different limit on number of entries.
cpl logs -a $APP_NAME --limit 100
# Uses a different loopback window.
cpl logs -a $APP_NAME --since 30min
```
### `maintenance`
Expand Down Expand Up @@ -311,6 +326,9 @@ cpl ps:stop -a $APP_NAME
# Stops a specific workload in app.
cpl ps:stop -a $APP_NAME -w $WORKLOAD_NAME
# Stops a specific replica of a workload.
cpl ps:stop -a $APP_NAME -w $WORKLOAD_NAME -r $REPLICA_NAME
```
### `ps:wait`
Expand All @@ -327,26 +345,43 @@ cpl ps:swait -a $APP_NAME -w $WORKLOAD_NAME
### `run`
- Runs one-off **_interactive_** replicas (analog of `heroku run`)
- Uses `Standard` workload type and `cpln exec` as the execution method, with CLI streaming
- If `fix_terminal_size` is `true` in the `.controlplane/controlplane.yml` file, the remote terminal size will be fixed to match the local terminal size (may also be overriden through `--terminal-size`)
> **IMPORTANT:** Useful for development where it's needed for interaction, and where network connection drops and
> task crashing are tolerable. For production tasks, it's better to use `cpl run:detached`.
- Runs one-off interactive or non-interactive replicas (analog of `heroku run`)
- Uses `Cron` workload type and either:
- - `cpln workload exec` for interactive mode, with CLI streaming
- - log async fetching for non-interactive mode
- The Dockerfile entrypoint is used as the command by default, which assumes `exec "${@}"` to be present,
and the args ["bash", "-c", cmd_to_run] are passed
- The entrypoint can be overriden through `--entrypoint`, which must be a single command or a script path that exists in the container,
and the args ["bash", "-c", cmd_to_run] are passed,
unless the entrypoint is `bash`, in which case the args ["-c", cmd_to_run] are passed
- Providing `--entrypoint none` sets the entrypoint to `bash` by default
- If `fix_terminal_size` is `true` in the `.controlplane/controlplane.yml` file,
the remote terminal size will be fixed to match the local terminal size (may also be overriden through `--terminal-size`)
```sh
# Opens shell (bash by default).
cpl run -a $APP_NAME
# Need to quote COMMAND if setting ENV value or passing args.
cpl run -a $APP_NAME -- 'LOG_LEVEL=warn rails db:migrate'
# Runs interactive command, keeps shell open, and stops job when exiting.
cpl run -a $APP_NAME --interactive -- rails c
# Runs command, displays output, and exits shell.
cpl run -a $APP_NAME -- ls /
cpl run -a $APP_NAME -- rails db:migrate:status
# Some commands are automatically detected as interactive, so no need to pass `--interactive`.
cpl run -a $APP_NAME -- bash
cpl run -a $APP_NAME -- rails console
cpl run -a $APP_NAME -- rails c
cpl run -a $APP_NAME -- rails dbconsole
cpl run -a $APP_NAME -- rails db
# Runs command and keeps shell open.
cpl run -a $APP_NAME -- rails c
# Runs non-interactive command, outputs logs, exits with the exit code of the command and stops job.
cpl run -a $APP_NAME -- rails db:migrate
# Runs non-iteractive command, detaches, exits with 0, and prints commands to:
# - see logs from the job
# - stop the job
cpl run -a $APP_NAME --detached -- rails db:migrate
# The command needs to be quoted if setting an env variable or passing args.
cpl run -a $APP_NAME -- 'SOME_ENV_VAR=some_value rails db:migrate'
# Uses a different image (which may not be promoted yet).
cpl run -a $APP_NAME --image appimage:123 -- rails db:migrate # Exact image name
Expand All @@ -358,47 +393,12 @@ cpl run -a $APP_NAME -w other-workload -- bash
# Overrides remote CPLN_TOKEN env variable with local token.
# Useful when superuser rights are needed in remote container.
cpl run -a $APP_NAME --use-local-token -- bash
```
### `run:cleanup`
- Deletes stale run workloads for an app
- Workloads are considered stale based on how many days since created
- `stale_run_workload_created_days` in the `.controlplane/controlplane.yml` file specifies the number of days after created that the workload is considered stale
- Works for both interactive workloads (created with `cpl run`) and non-interactive workloads (created with `cpl run:detached`)
- Will ask for explicit user confirmation of deletion
```sh
cpl run:cleanup -a $APP_NAME
```
### `run:detached`
- Runs one-off **_non-interactive_** replicas (close analog of `heroku run:detached`)
- Uses `Cron` workload type with log async fetching
- Implemented with only async execution methods, more suitable for production tasks
- Has alternative log fetch implementation with only JSON-polling and no WebSockets
- Less responsive but more stable, useful for CI tasks
- Deletes the workload whenever finished with success
- Deletes the workload whenever finished with failure by default
- Use `--no-clean-on-failure` to disable cleanup to help with debugging failed runs
```sh
cpl run:detached rails db:prepare -a $APP_NAME
# Need to quote COMMAND if setting ENV value or passing args.
cpl run:detached -a $APP_NAME -- 'LOG_LEVEL=warn rails db:migrate'
# Replaces the existing Dockerfile entrypoint with `bash`.
cpl run -a $APP_NAME --entrypoint none -- rails db:migrate
# Uses a different image (which may not be promoted yet).
cpl run:detached -a $APP_NAME --image appimage:123 -- rails db:migrate # Exact image name
cpl run:detached -a $APP_NAME --image latest -- rails db:migrate # Latest sequential image
# Uses a different workload than `one_off_workload` from `.controlplane/controlplane.yml`.
cpl run:detached -a $APP_NAME -w other-workload -- rails db:migrate:status
# Overrides remote CPLN_TOKEN env variable with local token.
# Useful when superuser rights are needed in remote container.
cpl run:detached -a $APP_NAME --use-local-token -- rails db:migrate:status
# Replaces the existing Dockerfile entrypoint.
cpl run -a $APP_NAME --entrypoint /app/alternative-entrypoint.sh -- rails db:migrate
```
### `setup-app`
Expand Down
8 changes: 4 additions & 4 deletions docs/migrating.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ cpl setup-app -a my-app-staging
cpl build-image -a my-app-staging --commit 456
# Prepare database.
cpl run:detached -a my-app-staging --image latest -- rails db:prepare
cpl run -a my-app-staging --image latest -- rails db:prepare
# Deploy latest image.
cpl deploy-image -a my-app-staging
Expand All @@ -113,7 +113,7 @@ cpl build-image -a my-app-staging --commit ABC
# Run database migrations (or other release tasks) with latest image, while app is still running on previous image.
# This is analogous to the release phase.
cpl run:detached -a my-app-staging --image latest -- rails db:migrate
cpl run -a my-app-staging --image latest -- rails db:migrate
# Deploy latest image.
cpl deploy-image -a my-app-staging
Expand Down Expand Up @@ -215,9 +215,9 @@ fi

# The `NEW_APP` env var that we exported above can be used to either reset or migrate the database before deploying.
if [ -n "${NEW_APP}" ]; then
cpl run:detached 'LOG_LEVEL=warn rails db:reset' -a ${APP_NAME} --image latest
cpl run -a ${APP_NAME} --image latest -- rails db:reset
else
cpl run:detached 'LOG_LEVEL=warn rails db:migrate_and_wait_replica' -a ${APP_NAME} --image latest
cpl run -a ${APP_NAME} --image latest -- rails db:migrate
fi
```

Expand Down
6 changes: 3 additions & 3 deletions examples/circleci.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ build-staging:
command: cpl build-image -a ${APP_NAME}
- run:
name: Database tasks
command: cpl run:detached -a ${APP_NAME} --image latest -- rails db:migrate
command: cpl run -a ${APP_NAME} --image latest -- rails db:migrate
- run:
name: Deploy image
command: cpl deploy-image -a ${APP_NAME}
Expand Down Expand Up @@ -75,9 +75,9 @@ build-review-app:
name: Database tasks
command: |
if [ -n "${NEW_APP}" ]; then
cpl run:detached -a ${APP_NAME} --image latest -- LOG_LEVEL=warn rails db:reset
cpl run -a ${APP_NAME} --image latest -- rails db:reset
else
cpl run:detached -a ${APP_NAME} --image latest -- LOG_LEVEL=warn rails db:migrate
cpl run -a ${APP_NAME} --image latest -- rails db:migrate
fi
- run:
name: Deploy image
Expand Down
12 changes: 4 additions & 8 deletions examples/controlplane.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ aliases:

# Control Plane offers the ability to use multiple locations.
# default_location is used for commands that require a location
# including `ps`, `run`, `run:detached`, `apply-template`.
# including `ps`, `run`, `apply-template`.
# This can be overridden with option --location=<location> and
# CPLN_LOCATION environment variable.
# TODO: Allow specification of multiple locations.
Expand Down Expand Up @@ -50,15 +50,15 @@ aliases:

# Workloads that are for the application itself and are using application Docker images.
# These are updated with the new image when running the `deploy-image` command,
# and are also used by the `info`, `ps:`, and `run:cleanup` commands in order to get all of the defined workloads.
# and are also used by the `info` and `ps:` commands in order to get all of the defined workloads.
# On the other hand, if you have a workload for Redis, that would NOT use the application Docker image
# and not be listed here.
app_workloads:
- rails
- sidekiq

# Additional "service type" workloads, using non-application Docker images.
# These are only used by the `info`, `ps:` and `run:cleanup` commands in order to get all of the defined workloads.
# These are only used by the `info` and `ps:` commands in order to get all of the defined workloads.
additional_workloads:
- redis
- postgres
Expand All @@ -68,7 +68,7 @@ aliases:
maintenance_workload: maintenance

# Fixes the remote terminal size to match the local terminal size
# when running the commands `cpl run` or `cpl run:detached`.
# when running `cpl run`.
fix_terminal_size: true

# Apps with a deployed image created before this amount of days will be listed for deletion
Expand All @@ -83,10 +83,6 @@ aliases:
# when running the command `cpl cleanup-images` (`image_retention_max_qty` takes precedence).
image_retention_days: 5

# Run workloads created before this amount of days will be listed for deletion
# when running the command `cpl run:cleanup`.
stale_run_workload_created_days: 2

apps:
my-app-staging:
# Use the values from the common section above.
Expand Down
Loading

0 comments on commit abbb3b6

Please sign in to comment.