Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Newest actions-runner-dind container failing to start #2143

Closed
7 tasks done
apenney opened this issue Jan 11, 2023 · 14 comments
Closed
7 tasks done

Newest actions-runner-dind container failing to start #2143

apenney opened this issue Jan 11, 2023 · 14 comments
Assignees
Labels
bug Something isn't working investigated Issue has been investigated and response provided

Comments

@apenney
Copy link

apenney commented Jan 11, 2023

Checks

Controller Version

2.300.2

Helm Chart Version

v0.21.0

CertManager Version

1.9.1

Deployment Method

Helm

cert-manager installation

.

Checks

  • This isn't a question or user support case (For Q&A and community support, go to Discussions. It might also be a good idea to contract with any of contributors and maintainers if your business is so critical and therefore you need priority support
  • I've read releasenotes before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes
  • My actions-runner-controller version (v0.x.y) does support the feature
  • I've already upgraded ARC (including the CRDs, see charts/actions-runner-controller/docs/UPGRADING.md for details) to the latest and it didn't fix the issue
  • I've migrated to the workflow job webhook event (if you using webhook driven scaling)

Resource Definitions

.

To Reproduce

.

Describe the bug

We were previously pulling :latest and upon the latest release (3 hours ago) we realized all our runners were failing with a looping:

Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

We ended up fixing by just pinning back to v2.299.1-ubuntu-20.04-aa6dab5 which allowed us to start things back up. We're going to do some troubleshooting to figure out what happened, but I wanted to get a placeholder issue in for anyone else who runs into this (assuming it's not hyper specific to our setup)

Describe the expected behavior

.

Whole Controller Logs

I would have these, but we already reverted in a panic.  I will edit this with logs as we test this without an outage :)

Whole Runner Pod Logs

❯ kubectl logs -n actions-runner-system runner-set-1-v8bgv-0
Using /etc/docker/daemon.json with the following content:
---
2023-01-11 17:11:32.219  NOTICE --- Runner init started with pid 10
{}
---
2023-01-11 17:11:32.221  DEBUG --- Starting Docker daemon
2023-01-11 17:11:32.222  DEBUG --- Waiting for processes to be running...
2023-01-11 17:11:32.228  DEBUG --- dockerd is running
2023-01-11 17:11:32.23  DEBUG --- Github endpoint URL https://github.com/
time="2023-01-11T17:11:32.253846102Z" level=info msg="Starting up"
time="2023-01-11T17:11:32.254880233Z" level=info msg="libcontainerd: started new containerd process" pid=48
time="2023-01-11T17:11:32.254914473Z" level=info msg="parsed scheme: \"unix\"" module=grpc
time="2023-01-11T17:11:32.254923082Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
time="2023-01-11T17:11:32.254937313Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}" module=grpc
time="2023-01-11T17:11:32.254948062Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2023-01-11T17:11:32Z" level=warning msg="containerd config version `1` has been deprecated and will be removed in containerd v2.0, please switch to version `2`, see https://github.com/containerd/containerd/blob/main/docs/PLUGINS.md#version-header"
time="2023-01-11T17:11:32.272722831Z" level=info msg="starting containerd" revision=1c90a442489720eec95342e1789ee8a5e1b9536f version=v1.6.9
time="2023-01-11T17:11:32.285107479Z" level=info msg="loading plugin \"io.containerd.content.v1.content\"..." type=io.containerd.content.v1
time="2023-01-11T17:11:32.285179499Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.aufs\"..." type=io.containerd.snapshotter.v1
time="2023-01-11T17:11:32.285360510Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.aufs\"..." error="aufs is not supported (modprobe aufs failed: exec: \"modprobe\": executable file not found in $PATH \"\"): skip plugin" type=io.containerd.snapshotter.v1
time="2023-01-11T17:11:32.285409639Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.btrfs\"..." type=io.containerd.snapshotter.v1
time="2023-01-11T17:11:32.285629839Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.btrfs\"..." error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs (ext4) must be a btrfs filesystem to be used with the btrfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
time="2023-01-11T17:11:32.285653159Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.devmapper\"..." type=io.containerd.snapshotter.v1
time="2023-01-11T17:11:32.285673009Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.devmapper" error="devmapper not configured"
time="2023-01-11T17:11:32.285685679Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.native\"..." type=io.containerd.snapshotter.v1
time="2023-01-11T17:11:32.285713929Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.overlayfs\"..." type=io.containerd.snapshotter.v1
time="2023-01-11T17:11:32.285880419Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.zfs\"..." type=io.containerd.snapshotter.v1
time="2023-01-11T17:11:32.286061079Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.zfs\"..." error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
time="2023-01-11T17:11:32.286085179Z" level=info msg="loading plugin \"io.containerd.metadata.v1.bolt\"..." type=io.containerd.metadata.v1
time="2023-01-11T17:11:32.286104799Z" level=warning msg="could not use snapshotter devmapper in metadata plugin" error="devmapper not configured"
time="2023-01-11T17:11:32.286119949Z" level=info msg="metadata content store policy set" policy=shared
time="2023-01-11T17:11:32.286244589Z" level=info msg="loading plugin \"io.containerd.differ.v1.walking\"..." type=io.containerd.differ.v1
time="2023-01-11T17:11:32.286272749Z" level=info msg="loading plugin \"io.containerd.event.v1.exchange\"..." type=io.containerd.event.v1
time="2023-01-11T17:11:32.286294999Z" level=info msg="loading plugin \"io.containerd.gc.v1.scheduler\"..." type=io.containerd.gc.v1
time="2023-01-11T17:11:32.286339399Z" level=info msg="loading plugin \"io.containerd.service.v1.introspection-service\"..." type=io.containerd.service.v1
time="2023-01-11T17:11:32.286382039Z" level=info msg="loading plugin \"io.containerd.service.v1.containers-service\"..." type=io.containerd.service.v1
time="2023-01-11T17:11:32.286413209Z" level=info msg="loading plugin \"io.containerd.service.v1.content-service\"..." type=io.containerd.service.v1
time="2023-01-11T17:11:32.286428269Z" level=info msg="loading plugin \"io.containerd.service.v1.diff-service\"..." type=io.containerd.service.v1
time="2023-01-11T17:11:32.286444019Z" level=info msg="loading plugin \"io.containerd.service.v1.images-service\"..." type=io.containerd.service.v1
time="2023-01-11T17:11:32.286849269Z" level=info msg="loading plugin \"io.containerd.service.v1.leases-service\"..." type=io.containerd.service.v1
time="2023-01-11T17:11:32.286874889Z" level=info msg="loading plugin \"io.containerd.service.v1.namespaces-service\"..." type=io.containerd.service.v1
time="2023-01-11T17:11:32.286886739Z" level=info msg="loading plugin \"io.containerd.service.v1.snapshots-service\"..." type=io.containerd.service.v1
time="2023-01-11T17:11:32.286896989Z" level=info msg="loading plugin \"io.containerd.runtime.v1.linux\"..." type=io.containerd.runtime.v1
time="2023-01-11T17:11:32.287039189Z" level=info msg="loading plugin \"io.containerd.runtime.v2.task\"..." type=io.containerd.runtime.v2
time="2023-01-11T17:11:32.287135319Z" level=info msg="loading plugin \"io.containerd.monitor.v1.cgroups\"..." type=io.containerd.monitor.v1
time="2023-01-11T17:11:32.288318789Z" level=info msg="loading plugin \"io.containerd.service.v1.tasks-service\"..." type=io.containerd.service.v1
time="2023-01-11T17:11:32.288439659Z" level=info msg="loading plugin \"io.containerd.grpc.v1.introspection\"..." type=io.containerd.grpc.v1
time="2023-01-11T17:11:32.288463599Z" level=info msg="loading plugin \"io.containerd.internal.v1.restart\"..." type=io.containerd.internal.v1
time="2023-01-11T17:11:32.288599719Z" level=info msg="loading plugin \"io.containerd.grpc.v1.containers\"..." type=io.containerd.grpc.v1
time="2023-01-11T17:11:32.288638139Z" level=info msg="loading plugin \"io.containerd.grpc.v1.content\"..." type=io.containerd.grpc.v1
time="2023-01-11T17:11:32.288678889Z" level=info msg="loading plugin \"io.containerd.grpc.v1.diff\"..." type=io.containerd.grpc.v1
time="2023-01-11T17:11:32.288707479Z" level=info msg="loading plugin \"io.containerd.grpc.v1.events\"..." type=io.containerd.grpc.v1
time="2023-01-11T17:11:32.288725609Z" level=info msg="loading plugin \"io.containerd.grpc.v1.healthcheck\"..." type=io.containerd.grpc.v1
time="2023-01-11T17:11:32.288752589Z" level=info msg="loading plugin \"io.containerd.grpc.v1.images\"..." type=io.containerd.grpc.v1
time="2023-01-11T17:11:32.288778829Z" level=info msg="loading plugin \"io.containerd.grpc.v1.leases\"..." type=io.containerd.grpc.v1
time="2023-01-11T17:11:32.288811849Z" level=info msg="loading plugin \"io.containerd.grpc.v1.namespaces\"..." type=io.containerd.grpc.v1
time="2023-01-11T17:11:32.288843749Z" level=info msg="loading plugin \"io.containerd.internal.v1.opt\"..." type=io.containerd.internal.v1
time="2023-01-11T17:11:32.289241939Z" level=info msg="loading plugin \"io.containerd.grpc.v1.snapshots\"..." type=io.containerd.grpc.v1
time="2023-01-11T17:11:32.289279949Z" level=info msg="loading plugin \"io.containerd.grpc.v1.tasks\"..." type=io.containerd.grpc.v1
time="2023-01-11T17:11:32.289295309Z" level=info msg="loading plugin \"io.containerd.grpc.v1.version\"..." type=io.containerd.grpc.v1
time="2023-01-11T17:11:32.289308079Z" level=info msg="loading plugin \"io.containerd.tracing.processor.v1.otlp\"..." type=io.containerd.tracing.processor.v1
time="2023-01-11T17:11:32.289334249Z" level=info msg="skip loading plugin \"io.containerd.tracing.processor.v1.otlp\"..." error="no OpenTelemetry endpoint: skip plugin" type=io.containerd.tracing.processor.v1
time="2023-01-11T17:11:32.289352409Z" level=info msg="loading plugin \"io.containerd.internal.v1.tracing\"..." type=io.containerd.internal.v1
time="2023-01-11T17:11:32.289407299Z" level=error msg="failed to initialize a tracing processor \"otlp\"" error="no OpenTelemetry endpoint: skip plugin"
time="2023-01-11T17:11:32.289751579Z" level=info msg=serving... address=/var/run/docker/containerd/containerd-debug.sock
time="2023-01-11T17:11:32.289864039Z" level=info msg=serving... address=/var/run/docker/containerd/containerd.sock.ttrpc
time="2023-01-11T17:11:32.289972099Z" level=info msg=serving... address=/var/run/docker/containerd/containerd.sock
time="2023-01-11T17:11:32.290008579Z" level=info msg="containerd successfully booted in 0.018235s"
time="2023-01-11T17:11:32.298704058Z" level=info msg="parsed scheme: \"unix\"" module=grpc
time="2023-01-11T17:11:32.298754438Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
time="2023-01-11T17:11:32.298776868Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}" module=grpc
time="2023-01-11T17:11:32.298790938Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2023-01-11T17:11:32.300021368Z" level=info msg="parsed scheme: \"unix\"" module=grpc
time="2023-01-11T17:11:32.300050778Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
time="2023-01-11T17:11:32.300067288Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}" module=grpc
time="2023-01-11T17:11:32.300075848Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2023-01-11T17:11:32.307291477Z" level=info msg="[graphdriver] using prior storage driver: overlay2"
time="2023-01-11T17:11:32.346200693Z" level=warning msg="Your kernel does not support cgroup blkio weight"
time="2023-01-11T17:11:32.346253873Z" level=warning msg="Your kernel does not support cgroup blkio weight_device"
time="2023-01-11T17:11:32.346465283Z" level=info msg="Loading containers: start."
time="2023-01-11T17:11:32.382252869Z" level=warning msg="Running iptables --wait -t nat -L -n failed with message: `# Warning: iptables-legacy tables present, use iptables-legacy to see them\niptables v1.8.7 (nf_tables): Could not fetch rule set generation id: Invalid argument`, error: exit status 4"
time="2023-01-11T17:11:32.411241586Z" level=info msg="stopping event stream following graceful shutdown" error="<nil>" module=libcontainerd namespace=moby
time="2023-01-11T17:11:32.411553556Z" level=info msg="stopping event stream following graceful shutdown" error="context canceled" module=libcontainerd namespace=plugins.moby
time="2023-01-11T17:11:32.411603316Z" level=info msg="stopping healthcheck following graceful shutdown" module=libcontainerd
2023-01-11 17:11:33.19  DEBUG --- Passing --ephemeral to config.sh to enable the ephemeral runner.
2023-01-11 17:11:33.293  DEBUG --- Configuring the runner.
failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to create NAT chain DOCKER: iptables failed: iptables -t nat -N DOCKER: iptables v1.8.7 (nf_tables): Could not fetch rule set generation id: Invalid argument

 (exit status 4)

--------------------------------------------------------------------------------
|        ____ _ _   _   _       _          _        _   _                      |
|       / ___(_) |_| | | |_   _| |__      / \   ___| |_(_) ___  _ __  ___      |
|      | |  _| | __| |_| | | | | '_ \    / _ \ / __| __| |/ _ \| '_ \/ __|     |
|      | |_| | | |_|  _  | |_| | |_) |  / ___ \ (__| |_| | (_) | | | \__ \     |
|       \____|_|\__|_| |_|\__,_|_.__/  /_/   \_\___|\__|_|\___/|_| |_|___/     |
|                                                                              |
|                       Self-hosted runner registration                        |
|                                                                              |
--------------------------------------------------------------------------------
Cannot configure the runner because it is already configured. To reconfigure the runner, run 'config.cmd remove' or './config.sh remove' first.
2023-01-11 17:11:33.597  DEBUG --- Runner successfully configured.
{
  "agentId": 43239,
  "agentName": "runner-set-1-v8bgv-0",
  "poolId": 1,
  "poolName": "Default",
  "ephemeral": true,
  "serverUrl": "https://pipelines.actions.githubusercontent.com/IyTJ9eRBwwk7bwfffZuenYsADKEOavLsGp7W6hzJYvCKDVBY9v",
  "gitHubUrl": "https://github.com/xxx",
  "workFolder": "/runner/_work"
2023-01-11 17:11:33.599  DEBUG --- Docker enabled runner detected and Docker daemon wait is enabled
2023-01-11 17:11:33.6  DEBUG --- Waiting until Docker is available or the timeout of 120 seconds is reached
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Additional Context

No response

@apenney apenney added bug Something isn't working needs triage Requires review from the maintainers labels Jan 11, 2023
@github-actions
Copy link
Contributor

Hello! Thank you for filing an issue.

The maintainers will triage your issue shortly.

In the meantime, please take a look at the troubleshooting guide for bug reports.

If this is a feature request, please review our contribution guidelines.

@kevholmes
Copy link

@apenney what image are you using? summerwind/actions-runner, dind or dind+rootless?

@apenney
Copy link
Author

apenney commented Jan 11, 2023

actions-runner-dind - I just did some testing (and added logs) - actions-runner-dind:ubuntu-20.04 works fine, actions-runner-dind:ubuntu-22.04 does not - so it's definitely isolated to 22.04 and not anything else.

@apenney
Copy link
Author

apenney commented Jan 11, 2023

failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to create NAT chain DOCKER: iptables failed: iptables -t nat -N DOCKER: iptables v1.8.7 (nf_tables): Could not fetch rule set generation id: Invalid argument - I assume it's this

@kevholmes
Copy link

Looks to be related to the upstream Debian change to nftables from iptables: https://wiki.debian.org/iptables. There's a workaround in there that might be worth trying and reporting back:

# update-alternatives --set iptables /usr/sbin/iptables-nft
# update-alternatives --set ip6tables /usr/sbin/ip6tables-nft
# update-alternatives --set arptables /usr/sbin/arptables-nft
# update-alternatives --set ebtables /usr/sbin/ebtables-nft

@pingping95
Copy link

Thanks so much.@apenney

I had a same problem and solved it

@ErdincWand
Copy link

I had the same error and needed to revert back to previous image and pipelines started to work again. We will wait for a fix.

Working image:
ghcr.io/actions-runner-controller/actions-runner-controller/actions-runner-dind:v2.299.1-ubuntu-20.04-acbce4b

@riccardosalamanna
Copy link

I wanted to report the same error, i have temporarily solved by using 20.04 as well

@xDmitriev
Copy link

Had the same issue with the latest image, and switched to the previous tag

@kevholmes
Copy link

kevholmes commented Jan 12, 2023

I've tested the following images in our GKE 1.25 environment running ARC 0.26.0 via helm chart version: 0.21.0.

ghcr.io/actions/actions-runner-controller/actions-runner-dind-rootless:v2.299.1-ubuntu-22.04-aa6dab5 ✅
ghcr.io/actions/actions-runner-controller/actions-runner-dind-rootless:ubuntu-22.04 ✅
ghcr.io/actions/actions-runner-controller/actions-runner-dind:ubuntu-22.04 ✅
summerwind/actions-runner-dind:v2.300.2-ubuntu-22.04-e71c646 ✅

The only way I can get those similar errors to occur is to have dockerdWithinRunnerContainer misconfigured for the image I've chosen. Eg I have it set to true when I'm using actions-runner and that will throw a similar error.

@Link- Link- self-assigned this Jan 13, 2023
@Link-
Copy link
Member

Link- commented Jan 13, 2023

The latest tag was applied to the image running ubuntu-22.04. This was an unintentional mistake due to the overhauling of our release workflows after the repository migration.

actions-runner-controller/releases#1 has addressed this problem and we're re-releasing the runner images with the appropriate tags.

Apologies for this!

@Link-
Copy link
Member

Link- commented Jan 13, 2023

Runner images have been tagged appropriately, which should resolve these problems. I'm going to keep this issue open pending a confirmation that you're all good 👍

@Link- Link- added investigated Issue has been investigated and response provided and removed needs triage Requires review from the maintainers labels Jan 13, 2023
@riccardosalamanna
Copy link

all solved for me, thanks

@Link- Link- closed this as completed Jan 18, 2023
@Nicarim
Copy link

Nicarim commented Feb 9, 2023

I fixed the issue with dind on ubuntu 22.04 by adding to my custom image this line

RUN update-alternatives --set iptables /usr/sbin/iptables-legacy

then dind started working correctly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working investigated Issue has been investigated and response provided
Projects
None yet
Development

No branches or pull requests

8 participants