Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[extension/cgroupruntime] Be aware of ECS task and CPU limits #36920

Merged
merged 13 commits into from
Jan 8, 2025
27 changes: 27 additions & 0 deletions .chloggen/check-ecs-metadata-cgroupruntimeextension.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: cgroupruntimeextension

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Implement ECS metadata retrieval for cgroupruntime extension.

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [36814]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:

# If your change doesn't affect end users or the exported elements of any package,
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: []
26 changes: 26 additions & 0 deletions extension/cgroupruntimeextension/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Contributing to the Cgroup Go runtime extension

In order to contribute to this extension, it might be useful to have a working local setup.

## Testing

Some Linux distributions don't run systemd under cgroupv2, to run the integration tests locally for this extension you can follow these steps.

Inside the extension folder, start a privileged docker container and share the code with the container

```bash
cd extension/cgroupruntimeextension
docker run -ti --privileged --cgroupns=host -v $(pwd):/workspace -w /workspace debian:bookworm-slim
```

Install the [Go version](https://go.dev/dl/) specified in the extension's [go.mod](./go.mod) and the GCC compiler to run the integration test. The following is an example command for Go `1.23.4` in and `amd64` system:

```bash
apt update && apt install -y wget sudo gcc && wget https://go.dev/dl/go1.23.4.linux-amd64.tar.gz && tar -C /usr/local -xzf go1.23.4.linux-amd64.tar.gz && export PATH=$PATH:/usr/local/go/bin && go version && rm go1.23.4.linux-amd64.tar.gz
```

Run the integration test

```bash
CGO_ENABLED=1 go test -v -exec sudo -race -timeout 360s -parallel 4 -tags=integration,""
```
6 changes: 5 additions & 1 deletion extension/cgroupruntimeextension/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

## Overview

The OpenTelemetry Cgroup Auto-Config Extension is designed to optimize Go runtime performance in containerized environments by automatically configuring GOMAXPROCS and GOMEMLIMIT based on the Linux cgroup filesystem. This extension leverages [automaxprocs](https://github.com/uber-go/automaxprocs) and [automemlimit](https://github.com/KimMachineGun/automemlimit) packages to dynamically adjust Go runtime variables, ensuring efficient resource usage aligned with container limits.
The OpenTelemetry Cgroup Auto-Config Extension is designed to optimize Go runtime performance in containerized environments by automatically configuring GOMAXPROCS and GOMEMLIMIT based on the Linux cgroup filesystem. This extension leverages [automaxprocs](https://github.com/uber-go/automaxprocs) or [gomaxecs](https://github.com/rdforte/gomaxecs) for AWS ECS Tasks and [automemlimit](https://github.com/KimMachineGun/automemlimit) packages to dynamically adjust Go runtime variables, ensuring efficient resource usage aligned with container limits.

## Configuration

Expand All @@ -40,3 +40,7 @@ extension:
enabled: true
ratio: 0.8
```

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for information on how to contribute to this extension.
9 changes: 7 additions & 2 deletions extension/cgroupruntimeextension/factory.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import (
"runtime/debug"

"github.com/KimMachineGun/automemlimit/memlimit"
gomaxecs "github.com/rdforte/gomaxecs/maxprocs"
"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/extension"
"go.uber.org/automaxprocs/maxprocs"
Expand Down Expand Up @@ -42,10 +43,14 @@ func createExtension(_ context.Context, set extension.Settings, cfg component.Co
cgroupConfig := cfg.(*Config)
return newCgroupRuntime(cgroupConfig, set.Logger,
func() (undoFunc, error) {
undo, err := maxprocs.Set(maxprocs.Logger(func(str string, params ...any) {
if gomaxecs.IsECS() {
return gomaxecs.Set(gomaxecs.WithLogger(func(str string, params ...any) {
set.Logger.Debug(fmt.Sprintf(str, params))
}))
}
return maxprocs.Set(maxprocs.Logger(func(str string, params ...any) {
set.Logger.Debug(fmt.Sprintf(str, params))
}))
return undoFunc(undo), err
},
func(ratio float64) (undoFunc, error) {
initial, err := memlimit.SetGoMemLimitWithOpts(memlimit.WithRatio(ratio))
Expand Down
3 changes: 2 additions & 1 deletion extension/cgroupruntimeextension/go.mod
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
module github.com/open-telemetry/opentelemetry-collector-contrib/extension/cgroupruntimeextension

go 1.22.0
go 1.22.4
r0mdau marked this conversation as resolved.
Show resolved Hide resolved

require (
github.com/KimMachineGun/automemlimit v0.7.0
github.com/containerd/cgroups/v3 v3.0.5
github.com/rdforte/gomaxecs v1.1.0
github.com/stretchr/testify v1.10.0
go.opentelemetry.io/collector/component v0.116.1-0.20241220212031-7c2639723f67
go.opentelemetry.io/collector/component/componenttest v0.116.1-0.20241220212031-7c2639723f67
Expand Down
2 changes: 2 additions & 0 deletions extension/cgroupruntimeextension/go.sum

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading