garm v0.1.4
Welcome to GARM version v0.1.4!
Documentation relevant to this version is available at: https://github.com/cloudbase/garm/tree/v0.1.4. The main
branch contains code that is still under development, and the documentation is relevant to that code. When deploying stable versions, please view the docs available under the relevant tag.
Before we dive into the highlights, there are a couple of things we need to mention.
Breaking change warning ⚠️
This version removes the LXD internal provider in favor of two new external providers:
This was done mainly due to the licensing change of LXD. If you're using LXD and are upgrading from previous versions, you must download the LXD external provider and configure it.
Another breaking (sort of) change is in the garm-cli
. The --force
flag in the runner remove
command, is no longer mandatory when removing a runner. Moreover, the --force
flag now ignores provider errors. In previous versions, if a provider erred when removing a runner, GARM would keep trying until it succeeded. The problem was that if a provider was misconfigured, we would be stuck with a runner in error state until re manually removed it from the database. The --force
flag allows us to remove a runner from GARM even if the provider returns an error.
Database upgrade ⚠️
This version will execute a database migration. While we try not to break anything when making database migrations, it's always a good idea to create a backup of your DB as well as any configuration files. If anything goes wrong, you can revert the changes.
To safely create a backup of you DB, you can use the sqlite3
CLI:
# Change the path to your DB to the one configured in your config.toml
sqlite3 /etc/garm/garm.db
.backup /path/to/backup/location/garm-backup.db
Highlights
This release is packed with optimizations, changes and a couple of nice features.
JIT runners
The main highlight of this release is the ability to use just-in-time self-hosted runners. To accommodate this change, GARM now has more robust metadata endpoints that can serve the needed JIT files. This will most likely be expanded in the future to include setup scripts for clouds where we have userdata size constraints.
JIT runners allows us to avoid sending a runner registration token over the wire. Registration tokens can be used to register multiple runners and have a validity of one hour. A bad actor may intercept this token and use it to register their own runners. With JIT runners, only one runner may use the credentials at any given point in time. The credentials are fetched from GARM by the runner when it spins up.
This feature can be disabled in the provider config by specifying:
[[provider]]
disable_jit_config = true
easily access controller info
We now have the ability to gain more insight into how GARM is configured. We've added a new command which can show us some info about the controller:
ubuntu@garm:~$ garm-cli controller-info show
+------------------------+----------------------------------------------------------------------------+
| FIELD | VALUE |
+------------------------+----------------------------------------------------------------------------+
| Controller ID | a4dd5f41-8e1e-42a7-af53-c0ba5ff6b0b3 |
| Hostname | garm |
| Metadata URL | https://garm.example.com/api/v1/metadata |
| Callback URL | https://garm.example.com/api/v1/callbacks |
| Webhook Base URL | https://garm.example.com/webhooks |
| Controller Webhook URL | https://garm.example.com/webhooks/a4dd5f41-8e1e-42a7-af53-c0ba5ff6b0b3 |
+------------------------+----------------------------------------------------------------------------+
Easy webhook installation
Installing webhooks for entities (repos, orgs, enterprises) can be a chore. In this version, we've added the ability to let GARM install the needed webhook, with the correct settings by running a simple command. To make use of this feature, the PAT you're using must have access to admin:org_hook
:
ubuntu@garm:~$ garm-cli org add \
--credentials org_token \
--name exampleOrg \
--install-webhook \
--random-webhook-secret
+----------------------+--------------------------------------+
| FIELD | VALUE |
+----------------------+--------------------------------------+
| ID | b90911e1-8727-4bb7-a1eb-96855d73a27b |
| Name | exampleOrg |
| Credentials | org_token |
| Pool manager running | true |
+----------------------+--------------------------------------+
This command will add the org exampleOrg
to your GARM controller and install a webhook for it with a random webhook secret. You can also install a webhook for an existing organization or repository:
ubuntu@garm:~$ garm-cli repo webhook install b90911e1-8727-4bb7-a1eb-96855d73a27b
+--------------+----------------------------------------------------------------------------+
| FIELD | VALUE |
+--------------+----------------------------------------------------------------------------+
| ID | 449734752 |
| URL | https://garm.example.com/webhooks/a4dd5f41-8e1e-42a7-af53-c0ba5ff6b0b3 |
| Events | [workflow_job] |
| Active | true |
| Insecure SSL | false |
+--------------+----------------------------------------------------------------------------+
To view the status of a webhook:
ubuntu@garm:~$ garm-cli repo webhook show b90911e1-8727-4bb7-a1eb-96855d73a27b
+--------------+----------------------------------------------------------------------------+
| FIELD | VALUE |
+--------------+----------------------------------------------------------------------------+
| ID | 449734752 |
| URL | https://garm.example.com/webhooks/a4dd5f41-8e1e-42a7-af53-c0ba5ff6b0b3 |
| Events | [workflow_job] |
| Active | true |
| Insecure SSL | false |
+--------------+----------------------------------------------------------------------------+
You can also uninstall a webhook:
ubuntu@garm:~$ garm-cli repo webhook uninstall b90911e1-8727-4bb7-a1eb-96855d73a27b
Webhooks installed by the above commands are always namespaced to the controller ID you get when you run garm-cli controller-info show
. This way, when we remove a webhook, we don't accidentally remove someone else's hook. You can also manually install we webhook just like before. You don't need to namespace it to the controller ID. GARM will see the webhook and let you know if the hook is already installed.
Removing a repo or org will also clean up the webhook if it was namespaced to our controller. You can opt to keep the hook by passing the --keep-webhook
flag.
Webhook installation is not available for enterprises.
Structured logging
GARM has now switched to the slog
standard package for structured logging. As part of this change, we now have a dedicated [logging]
config section where you can set the log level, log format, enable the log streamer, etc. Check out the sample config for more info.
Other updates
There are a lot of bug fixes and stability updates that should make this version behave better in relation to the GitHub API.
What's Changed
- Update to latest version of garm-provider-common and update docs by @gabriel-samfira in #144
- Add GitHub workflow for integration tests by @mihaelabalutoiu in #145
- Add upload artifacts and log instance details on timeout exceeded by @mihaelabalutoiu in #146
- Add workflow dispatch trigger by @gabriel-samfira in #147
- Set pipefail to the script and log org/repo details on timeout exceeded by @mihaelabalutoiu in #148
- Cleaning up leftover runners for
org/repo
by @mihaelabalutoiu in #149 - Add controller info by @gabriel-samfira in #150
- Log
orgPool/repoPool
details on timeout exceeded by @mihaelabalutoiu in #151 - Fix timeout logic by @mihaelabalutoiu in #152
- Small fixes for the docs by @mihaelabalutoiu in #153
- Enable Windows builds by @gabriel-samfira in #156
- Add webhook management for repositories and organizations by @gabriel-samfira in #154
- Fix e2e secrets generation by @ionutbalutoiu in #162
- Add webhooks integration tests for
organization
andrepository
by @mihaelabalutoiu in #161 - Use apg to generate passwords by @gabriel-samfira in #164
- Fix nil pointer dereference by @gabriel-samfira in #165
- Fix TLS client bug by @gabriel-samfira in #166
- Refactor integration E2E tests by @ionutbalutoiu in #167
- Fix
waitPoolRunningIdleInstances
function by @ionutbalutoiu in #168 - Fix garm pool manager startup by @gabriel-samfira in #169
- Properly set runner group when creating a pool by @gabriel-samfira in #170
- Optimize
waitPoolRunningIdleInstances
func by @ionutbalutoiu in #171 - Add relation to jobs by @gabriel-samfira in #172
- Switch to seal unseal by @gabriel-samfira in #173
- Add root CA bundle metadata URL by @gabriel-samfira in #174
- Cleaning up leftover Github webhooks for
org/repo
by @mihaelabalutoiu in #175 - Update go-github and garm-provider-common by @gabriel-samfira in #179
- Add jit config by @gabriel-samfira in #163
- feat: add new metrics by @bavarianbidi in #180
- Update garm-provider-common by @gabriel-samfira in #182
- Add force delete runner by @gabriel-samfira in #183
- feat: passthrough additional env vars to provider bin by @bavarianbidi in #188
- Add option to disable JIT config by @gabriel-samfira in #189
- Add the k8s provider to the list by @gabriel-samfira in #190
- Add some more info to README by @gabriel-samfira in #191
- Prevent abusing the GH API by @gabriel-samfira in #193
- Remove the LXD internal provider by @gabriel-samfira in #194
- Update deps by @gabriel-samfira in #195
- Export required variables by @gabriel-samfira in #196
- Copy provider config in garm folder by @gabriel-samfira in #197
- Create needed folders before use by @gabriel-samfira in #198
- Add system-info instance callback by @gabriel-samfira in #200
- Switch to slog by @gabriel-samfira in #201
- Fix log streamer and cleanup code by @gabriel-samfira in #202
- Small adjustments by @gabriel-samfira in #203
- Add some logging by @gabriel-samfira in #205
- More strict instance token checks by @gabriel-samfira in #206
- Update readme by @gabriel-samfira in #207
- Add GCP to the list of providers by @gabriel-samfira in #208
- Fix typos in garm-cli by @gabriel-samfira in #209
- Add new providers by @gabriel-samfira in #210
New Contributors
- @bavarianbidi made their first contribution in #180
Full Changelog: v0.1.3...v0.1.4