Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop #42

Merged
merged 13 commits into from
Jan 9, 2024
Merged

Develop #42

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
9519e75
refactor: disable storybook controls for certain arg types.
thediveo Nov 17, 2023
f80e372
refactor: more storybook component stories
thediveo Nov 18, 2023
61304d5
chore: CI, codeql js/ts scanning (#41)
thediveo Nov 20, 2023
ad4ea9e
fix: parsing PIDs in combination with PIDType casts
thediveo Nov 21, 2023
2025fb2
wip: more component stories
thediveo Nov 21, 2023
0707645
feat: podman container icon
thediveo Jan 5, 2024
69e5935
ui: new help chapter on container and container group symbols
thediveo Jan 5, 2024
463cf46
doc: update README
thediveo Jan 5, 2024
7db877e
chore: upgrade deps, especially turtlefinder
thediveo Jan 6, 2024
560e22b
fix: use cri test img import from whalewatcher instead of turtlefinder
thediveo Jan 6, 2024
1f8b9c3
chore: upgrade vite
thediveo Jan 7, 2024
0eaab94
fix: use engine's PPID for container PID translation when engine PID …
thediveo Jan 7, 2024
2d07c99
ci: action chores
thediveo Jan 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions .github/workflows/buildandrelease.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,15 @@ jobs:
packages: write
steps:
- name: Check out the repo
uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # pin@v3
uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # pin@v3

- name: Set up QEMU
uses: docker/setup-qemu-action@2b82ce82d56a2a04d2637cd93a637ae1b359c0a7 # pin@v2
with:
platforms: ${{ env.PLATFORMS }}

- name: Set up Docker buildx
uses: docker/setup-buildx-action@6a58db7e0d21ca03e6c44877909e80e45217eed2 # pin@v2
uses: docker/setup-buildx-action@885d1462b80bc1c1c7f0b00334ad271f09369c55 # pin@v2

- name: Log into the container registry
uses: docker/login-action@465a07811f14bebb1938fbed4728c6a1ff8901fc # pin@v2
Expand All @@ -37,7 +37,7 @@ jobs:
password: ${{ secrets.GITHUB_TOKEN }}

- name: Docker metadata
uses: docker/metadata-action@2c0bd771b40637d97bf205cbccdd294a32112176 # pin@v4
uses: docker/metadata-action@818d4b7b91585d195f67373fd9cb0332e31a7175 # pin@v4
id: metadata # later referenced as "steps.metadata."
with:
images: ghcr.io/${{ github.repository }}
Expand All @@ -62,7 +62,7 @@ jobs:
echo "$EOF" >> $GITHUB_ENV

- name: Build and push Docker image
uses: docker/build-push-action@3b5e8027fcad23fda98b2e3ac259d8d67585f671 # pin@v4
uses: docker/build-push-action@0a97817b6ade9f46837855d676c4cca3a2471fc9 # pin@v4
with:
context: .
file: ./deployments/lxkns/Dockerfile
Expand All @@ -76,4 +76,5 @@ jobs:
build-contexts: |
webappsrc=./web/lxkns
${{ env.BUILDCONTEXTS }}
outputs: type=image,name=target,annotation-index.org.opencontainers.image.description=Linux kernel namespaces discovery
outputs: type=image,name=target,annotation-index.org.opencontainers.image.description=Linux
kernel namespaces discovery
4 changes: 2 additions & 2 deletions .github/workflows/buildandtest.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,13 +25,13 @@ jobs:
sudo docker -H unix:///proc/1/root/run/docker.sock version

- name: Set up Go ${{matrix.go}}
uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # pin@v4
uses: actions/setup-go@93397bea11091df50f3d7e59dc26a7711a8bcfbe # pin@v4
with:
go-version: ${{matrix.go}}
id: go

- name: Check out code into the Go module directory
uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # pin@v3
uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # pin@v3

- name: Get dependencies
run: go get -v -t -d ./...
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/buildxkcd2347.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,12 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # pin@v3
uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # pin@v3

- name: test build
run: |
rm -f .yarnrc.yml
corepack enable
yarn set version berry
yarn config set nodeLinker node-modules
yarn workspaces focus --production
Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/codeql-analysis.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,19 +24,19 @@ jobs:
strategy:
fail-fast: false
matrix:
language: [ 'go' ]
language: [ 'go', 'javascript' ]

steps:
- name: Checkout repository
uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # pin@v3
uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # pin@v3

- name: Initialize CodeQL
uses: github/codeql-action/init@83f0fe6c4988d98a455712a27f0255212bba9bd4 # pin@v2
uses: github/codeql-action/init@8b7fcbfac2aae0e6c24d9f9ebd5830b1290b18e4 # pin@v2
with:
languages: ${{ matrix.language }}

- name: Autobuild
uses: github/codeql-action/autobuild@83f0fe6c4988d98a455712a27f0255212bba9bd4 # pin@v2
uses: github/codeql-action/autobuild@8b7fcbfac2aae0e6c24d9f9ebd5830b1290b18e4 # pin@v2

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@83f0fe6c4988d98a455712a27f0255212bba9bd4 # pin@v2
uses: github/codeql-action/analyze@8b7fcbfac2aae0e6c24d9f9ebd5830b1290b18e4 # pin@v2
18 changes: 11 additions & 7 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,6 @@
![file descriptors](https://img.shields.io/badge/file%20descriptors-not%20leaking-success)
[![Go Report Card](https://goreportcard.com/badge/github.com/thediveo/lxkns)](https://goreportcard.com/report/github.com/thediveo/lxkns)

> **NEW:** lxkns now leverages [(Siemens OSS) Turtlefinder
> technology](https://github.com/siemens/turtlefinder) to autodetect container
> engines even in hierarchical configurations, such as Kubernetes-in-Docker and
> Docker Desktop on WSL2.

## Quick Start

Expand All @@ -40,12 +36,19 @@ namespaces, as well as mount points with their hierarchies.
## Overview

`lxkns` discovers...
- Linux namespaces in almost every nook and cranny of your hosts (open file
descriptors, bind-mounts, processes, and now even tasks) – please see the table below,
- Linux namespaces in almost every nook and cranny of your hosts (from open file
descriptors, bind-mounts, processes, and now even tasks and from open sockets)
– please see the table below,
- the mount points inside mount namespaces (correctly representing
"overmounts").
- container workloads: these are then related to the underlying Linux
namespaces.
- `lxkns` now leverages [(Siemens OSS) Turtlefinder
technology](https://github.com/siemens/turtlefinder) to autodetect container
engines even in hierarchical configurations, such as Kubernetes-in-Docker
and Docker Desktop on WSL2. Also, (socket-activated) podman detection has
finally landed in Turtlefinder, and in turn also in `lxkns`.


| | Where? | `lsns` | `lxkns` |
| --- | --- | :---: | :---: |
Expand All @@ -60,7 +63,8 @@ namespaces, as well as mount points with their hierarchies.
The following container engine types are supported:
- Docker,
- plain containerd,
- CRI Evented PLEG: containerd, CRI-O.
- CRI Evented PLEG: containerd, CRI-O,
- podman (via its Docker-compatible API only).

The `lxkns` discovery engine can be operated as a stand-alone REST service with
additional web UI. Alternatively, it can be embedded/integrated into other
Expand Down
2 changes: 1 addition & 1 deletion decorator/kuhbernetes/cri/decorator_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,12 @@ import (
"github.com/google/uuid"
"github.com/ory/dockertest/v3"
"github.com/ory/dockertest/v3/docker"
"github.com/siemens/turtlefinder/detector/crio/test/img"
"github.com/thediveo/lxkns/containerizer/whalefriend"
"github.com/thediveo/lxkns/decorator/kuhbernetes"
"github.com/thediveo/lxkns/model"
"github.com/thediveo/lxkns/test/matcher"
criengine "github.com/thediveo/whalewatcher/engineclient/cri"
"github.com/thediveo/whalewatcher/engineclient/cri/test/img"
"github.com/thediveo/whalewatcher/test"
"github.com/thediveo/whalewatcher/watcher"
runtime "k8s.io/cri-api/pkg/apis/runtime/v1"
Expand Down
3 changes: 0 additions & 3 deletions decorator/kuhbernetes/cri/package_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,11 @@ package cri

import (
"testing"
"time"

. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)

var slowSpec = NodeTimeout(30 * time.Second)

func TestK8sContainerdDecorator(t *testing.T) {
RegisterFailHandler(Fail)
RunSpecs(t, "lxkns/decorator/kuhbernetes/cri package")
Expand Down
2 changes: 1 addition & 1 deletion defs_version.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion deployments/lxkns/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

ARG ALPINE_VERSION=3.18
ARG ALPINE_PATCH=4
ARG GO_VERSION=1.21.3
ARG GO_VERSION=1.21.4
ARG NODE_VERSION=21

# 0th stage: https://github.com/tonistiigi/xx/blob/master/README.md
Expand Down
3 changes: 2 additions & 1 deletion discover/discovery_bindmount.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,8 @@ func refString(mntns model.Namespace, r *Result) string {
for _, ref := range refs {
if strings.HasPrefix(ref, "/proc/") {
if f := strings.SplitN(ref, "/", 4); len(f) >= 3 {
if pid, err := strconv.ParseUint(f[2], 10, 32); err == nil {
// PIDs are unsigned, but passed as int32...
if pid, err := strconv.ParseUint(f[2], 10, 31); err == nil {
if proc := r.Processes[model.PIDType(pid)]; proc != nil {
s = append(s, fmt.Sprintf("%s[=%s]", ref, proc.Name))
}
Expand Down
16 changes: 16 additions & 0 deletions discover/discovery_containers.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,22 @@ func discoverContainers(result *Result) {
if !ok {
if engineProc, ok := result.Processes[container.Engine.PID]; ok {
enginePIDns = engineProc.Namespaces[model.PIDNS]
} else if container.Engine.PPIDHint != 0 {
// This is a newly socket-activated engine that isn't yet
// included in the process tree – that process tree that
// ironically lead to the detection of the socket activator and
// then activation of that container engine. As we cannot change
// the past discovery some kind soul – a turtle, perchance? –
// might have passed us a hint about the engine's parent process
// PID. This parent process's PID namespace should be the same
// as the container engine, so it should be good for container
// PID translation.
//
// This deserves a badge: [COMMENTOR] ... rhymes with
// "tormentor" *snicker*
if parentProc, ok := result.Processes[container.Engine.PPIDHint]; ok {
enginePIDns = parentProc.Namespaces[model.PIDNS]
}
}
// Cache even unsuckcessful engine PID namespace lookups.
enginesPIDns[container.Engine] = enginePIDns
Expand Down
3 changes: 2 additions & 1 deletion discover/discovery_fd.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,8 @@ func scanFd(_ species.NamespaceType, procfs string, fakeprocfs bool, result *Res
// namespaceOfSocket returns the network namespace a particular socket fd (of
// the specified process) is connected to.
func namespaceOfSocket(pidfd int, fdname string) (species.NamespaceID, species.NamespaceType) {
fdno, err := strconv.ParseUint(fdname, 10, 32)
// PIDs are unsigned, but passed as int32...
fdno, err := strconv.ParseUint(fdname, 10, 31)
if err != nil {
return species.NoneID, species.NaNS
}
Expand Down
3 changes: 2 additions & 1 deletion discover/ref.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ func PIDfromPath(path string) model.PIDType {
if idx := strings.Index(path, "/"); idx >= 0 {
pidfield = path[:idx]
}
pid, err := strconv.ParseUint(pidfield, 10, 32)
// PIDs are unsigned, but passed as int32...
pid, err := strconv.ParseUint(pidfield, 10, 31)
if err != nil {
return 0
}
Expand Down
47 changes: 25 additions & 22 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ go 1.20

require (
github.com/PaesslerAG/jsonpath v0.1.1
github.com/containerd/containerd v1.7.8
github.com/getkin/kin-openapi v0.120.0
github.com/containerd/containerd v1.7.11
github.com/getkin/kin-openapi v0.122.0
github.com/gorilla/mux v1.8.1
github.com/muesli/termenv v0.15.2
github.com/onsi/ginkgo/v2 v2.13.1
github.com/onsi/ginkgo/v2 v2.13.2
github.com/onsi/gomega v1.30.0
github.com/ory/dockertest/v3 v3.10.0
github.com/siemens/turtlefinder v1.0.2
github.com/siemens/turtlefinder v1.1.0
github.com/sirupsen/logrus v1.9.3
github.com/spf13/cobra v1.8.0
github.com/thediveo/enumflag/v2 v2.0.5
Expand All @@ -26,20 +26,22 @@ require (
github.com/thediveo/spaserve v1.0.2
github.com/thediveo/success v1.0.2
github.com/thediveo/testbasher v1.0.8
github.com/thediveo/whalewatcher v0.10.2
golang.org/x/sys v0.14.0
github.com/thediveo/whalewatcher v0.11.0
golang.org/x/sys v0.16.0
golang.org/x/text v0.14.0
gopkg.in/yaml.v3 v3.0.1
k8s.io/cri-api v0.28.3
k8s.io/cri-api v0.28.5
)

require (
github.com/AdaLogics/go-fuzz-headers v0.0.0-20230811130428-ced1acdcaa24 // indirect
github.com/AdamKorcz/go-118-fuzz-build v0.0.0-20230306123547-8075edf89bb0 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/containerd/log v0.1.0 // indirect
github.com/containerd/typeurl/v2 v2.1.1 // indirect
github.com/docker/distribution v2.8.2+incompatible // indirect
github.com/felixge/httpsnoop v1.0.3 // indirect
github.com/go-logr/logr v1.3.0 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-sql-driver/mysql v1.7.0 // indirect
Expand All @@ -52,20 +54,21 @@ require (
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826 // indirect
github.com/perimeterx/marshmallow v1.1.5 // indirect
github.com/prometheus/procfs v0.9.0 // indirect
go.opentelemetry.io/otel v1.16.0 // indirect
go.opentelemetry.io/otel/metric v1.16.0 // indirect
go.opentelemetry.io/otel/trace v1.16.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.45.0 // indirect
go.opentelemetry.io/otel v1.19.0 // indirect
go.opentelemetry.io/otel/metric v1.19.0 // indirect
go.opentelemetry.io/otel/trace v1.19.0 // indirect
go.uber.org/goleak v1.2.1 // indirect
golang.org/x/mod v0.14.0 // indirect
golang.org/x/tools v0.15.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20230822172742-b8732ec3820d // indirect
golang.org/x/tools v0.16.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20231002182017-d307bd883b97 // indirect
gotest.tools/v3 v3.4.0 // indirect
)

require (
github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 // indirect
github.com/Microsoft/go-winio v0.6.1 // indirect
github.com/Microsoft/hcsshim v0.11.1 // indirect
github.com/Microsoft/hcsshim v0.11.4 // indirect
github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5 // indirect
github.com/PaesslerAG/gval v1.0.0 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
Expand All @@ -88,7 +91,7 @@ require (
github.com/golang/protobuf v1.5.3 // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
github.com/google/uuid v1.4.0
github.com/google/uuid v1.5.0
github.com/imdario/mergo v0.3.16 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/klauspost/compress v1.16.6 // indirect
Expand All @@ -108,22 +111,22 @@ require (
github.com/opencontainers/selinux v1.11.0 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/rivo/uniseg v0.4.4 // indirect
github.com/samber/lo v1.38.1
github.com/samber/lo v1.39.0
github.com/spf13/pflag v1.0.5 // indirect
github.com/thediveo/caps v0.10.0
github.com/thediveo/go-plugger/v3 v3.0.1
github.com/thediveo/caps v0.10.1
github.com/thediveo/go-plugger/v3 v3.1.0
github.com/thediveo/once v0.9.1
github.com/vishvananda/netlink v1.2.1-beta.2.0.20230206183746-70ca0345eede // indirect
github.com/vishvananda/netns v0.0.0-20210104183010-2eb08e3e575f // indirect
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
github.com/xeipuuv/gojsonschema v1.2.0 // indirect
go.opencensus.io v0.24.0 // indirect
golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa
golang.org/x/net v0.18.0 // indirect
golang.org/x/sync v0.5.0 // indirect
google.golang.org/genproto v0.0.0-20230822172742-b8732ec3820d // indirect
google.golang.org/grpc v1.59.0 // indirect
golang.org/x/exp v0.0.0-20240103183307-be819d1f06fc
golang.org/x/net v0.19.0 // indirect
golang.org/x/sync v0.6.0 // indirect
google.golang.org/genproto v0.0.0-20231002182017-d307bd883b97 // indirect
google.golang.org/grpc v1.60.1 // indirect
google.golang.org/protobuf v1.31.0 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
k8s.io/client-go v0.28.3 // indirect
Expand Down
Loading
Loading