From c4ba0b2bdd1d18170c934fce6b37eb179892d56a Mon Sep 17 00:00:00 2001 From: Jacob Aronoff Date: Wed, 14 Feb 2024 13:21:38 -0500 Subject: [PATCH 1/2] Prep release bump for the arrow col --- README.md | 10 +++---- arrow/Dockerfile | 2 +- arrow/otelcolarrow-build.yaml | 54 +++++++++++++++++------------------ 3 files changed, 33 insertions(+), 33 deletions(-) diff --git a/README.md b/README.md index e2c1d4d..c566ded 100644 --- a/README.md +++ b/README.md @@ -7,11 +7,11 @@ This is the repository for recommended [Helm](https://helm.sh/) charts for runni ## Charts * [otel-cloud-stack](https://github.com/lightstep/prometheus-k8s-opentelemetry-collector/tree/main/charts/otel-cloud-stack) - **Recommended** chart for sending Kubernetes metrics to ServiceNow Cloud Observability using OpenTelemetry-native metric collection and the OpenTelemetry Operator. -* [kube-otel-stack](https://github.com/lightstep/prometheus-k8s-opentelemetry-collector/tree/main/charts/kube-otel-stack) - Drop in replacement for [kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack), which uses the same configuration for scraping Prometheus exporters and forwarding metrics to Lightstep using the OpenTelemetry Operator. Use this chart if you are looking to compare Kubernetes monitoring in Prometheus with Kubernetes monitoring using ServiceNow Cloud Observability. +* [kube-otel-stack](https://github.com/lightstep/prometheus-k8s-opentelemetry-collector/tree/main/charts/kube-otel-stack) - Drop in replacement for [kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack), which uses the same configuration for scraping Prometheus exporters and forwarding metrics to Lightstep using the OpenTelemetry Operator. Use this chart if you are looking to compare Kubernetes monitoring in Prometheus with Kubernetes monitoring using ServiceNow Cloud Observability. ## Arrow Usage -> [!NOTE] +> [!NOTE] > Arrow usage is in beta, please use at your own risk. Reach out if you have any issues. In order to use an arrow trace collector, you can use (1) the prebuilt image available via the Github Container Registry (GHCR) or you may (2) build your own custom image. @@ -19,13 +19,13 @@ In order to use an arrow trace collector, you can use (1) the prebuilt image av ### 1. Use the prebuilt Docker image 1. We have built a Docker image using the recommended [build config](https://github.com/lightstep/otel-collector-charts/blob/main/arrow/otelcolarrow-build.yaml) 2. This Docker [image](https://github.com/lightstep/otel-collector-charts/pkgs/container/otel-collector-charts%2Fotelarrowcol-experimental) can be pulled by running: `docker pull ghcr.io/lightstep/otel-collector-charts/otelarrowcol-experimental:latest` -3. You can use the collector config (`/arrow/config/gateway-config.yaml`) by running: +3. You can use the collector config (`/arrow/config/gateway-collector.yaml`) by running: `docker run -it -v $(PWD)/config/:/config --entrypoint /otelarrowcol ghcr.io/lightstep/otel-collector-charts/otelarrowcol-experimental:latest --config=/config/gateway-collector.yaml` ### 2. Build your own custom image -1. We have supplied a collector builder config below. -2. Once an image is a available, simply apply your desired helm chart with the values.yaml AND the arrow.yaml in the respective chart. +1. We have supplied a collector builder config below. +2. Once an image is a available, simply apply your desired helm chart with the values.yaml AND the arrow.yaml in the respective chart. 3. Make sure to replace the image in arrow.yaml with your custom built image. ## Build configurations diff --git a/arrow/Dockerfile b/arrow/Dockerfile index 288574e..0f4d678 100644 --- a/arrow/Dockerfile +++ b/arrow/Dockerfile @@ -6,7 +6,7 @@ ENV CGO_ENABLED=0 # Note the version MUST MATCH otelarrowcol-build.yaml # Future optimization - curl the release. -RUN go install go.opentelemetry.io/collector/cmd/builder@v0.91.0 +RUN go install go.opentelemetry.io/collector/cmd/builder@v0.94.1 # This command generates main.go, go.mod but does not update deps. RUN builder --config=/otelarrowcol/otelcolarrow-build.yaml diff --git a/arrow/otelcolarrow-build.yaml b/arrow/otelcolarrow-build.yaml index 25544e3..0ce442c 100644 --- a/arrow/otelcolarrow-build.yaml +++ b/arrow/otelcolarrow-build.yaml @@ -37,13 +37,13 @@ dist: # description: My Organization telemetry collector # name: otelarrowcol - version: 0.10.0 + version: 0.17.0 description: ServiceNow Cloud Observability OpenTelemetry Protocol with Apache Arrow gateway collector # This indicates which version of the core collector components as # well as the builder version. At the time of this writing, we # recommend using at least the current release. - otelcol_version: 0.91.0 + otelcol_version: 0.94.1 # We recommend building in both the OTLP exporter and the OTel-Arrow # exporter. @@ -51,7 +51,7 @@ exporters: # This is the core OpenTelemetry Protocol with Apache Arrow exporter, # recommended for exporting to ServiceNow Cloud Observability using # either OTel-Arrow or standard OTLP over gRPC. - - gomod: github.com/open-telemetry/otel-arrow/collector v0.13.0 + - gomod: github.com/open-telemetry/otel-arrow/collector v0.17.0 import: github.com/open-telemetry/otel-arrow/collector/exporter/otelarrowexporter # The following components may be useful for debugging. @@ -59,55 +59,55 @@ exporters: # As an alternate to the OTel-Arrow exporter, we recommend building # with the core OTLP exporter as a fallback. These components use # compatible configuration. - - gomod: go.opentelemetry.io/collector/exporter/otlpexporter v0.91.0 + - gomod: go.opentelemetry.io/collector/exporter/otlpexporter v0.94.1 # In case OTLP/HTTP export is required, as opposed to the two # gRPC options above. - - gomod: go.opentelemetry.io/collector/exporter/otlphttpexporter v0.91.0 + - gomod: go.opentelemetry.io/collector/exporter/otlphttpexporter v0.94.1 # In case you want to record an OTLP telemetry session to a JSON # file, we recommend this additional utility. See also the # obfuscation processor, listed below. The OTel-Arrow provides # offline tools that can help explain poor compression performance # using inputs generated by this exporter, for example. - - gomod: github.com/open-telemetry/otel-arrow/collector v0.13.0 + - gomod: github.com/open-telemetry/otel-arrow/collector v0.17.0 import: github.com/open-telemetry/otel-arrow/collector/exporter/fileexporter # The debug exporter, useful for printing telemetry to the console. - - gomod: go.opentelemetry.io/collector/exporter/debugexporter v0.91.0 + - gomod: go.opentelemetry.io/collector/exporter/debugexporter v0.94.1 receivers: # The OTel-Arrow receiver supports multiple protocols including OTel-Arrow, OTLP gRPC, # OTLP HTTP/proto, and OTLP HTTP/json. We recommend using this component to receive # OTLP (all forms) and OTel-Arrow in a single component. - # + # # This enables other OTel Collectors or experimental SDKs that support OTel-Arrow to # send to this collector, such as this one for the OTel-Go Trace SDK: # https://github.com/lightstep/otel-launcher-go/tree/main/lightstep/sdk/trace/exporters/otlp/otelcol - - gomod: github.com/open-telemetry/otel-arrow/collector v0.13.0 + - gomod: github.com/open-telemetry/otel-arrow/collector v0.17.0 import: github.com/open-telemetry/otel-arrow/collector/receiver/otelarrowreceiver # To support receiving OTLP/HTTP. - - gomod: go.opentelemetry.io/collector/receiver/otlpreceiver v0.91.0 + - gomod: go.opentelemetry.io/collector/receiver/otlpreceiver v0.94.1 # You may wish to enable other receivers from the Collector-Contrib repository or # elsewhere. Here are some that might be useful: # - # - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.91.0 - # - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver v0.91.0 - # - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/kubeletstatsreceiver v0.91.0 - # - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/k8sclusterreceiver v0.91.0 + # - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.94.0 + # - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver v0.94.0 + # - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/kubeletstatsreceiver v0.94.0 + # - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/k8sclusterreceiver v0.94.0 processors: # We recommend use of the OTel-Arrow concurrent batch processor. - - gomod: github.com/open-telemetry/otel-arrow/collector v0.13.0 + - gomod: github.com/open-telemetry/otel-arrow/collector v0.17.0 import: github.com/open-telemetry/otel-arrow/collector/processor/concurrentbatchprocessor # We recommend building with the follow processor for obfuscation, in case you want to # record telemetry sessions for offline analysis. - - gomod: github.com/open-telemetry/otel-arrow/collector v0.13.0 + - gomod: github.com/open-telemetry/otel-arrow/collector v0.17.0 import: github.com/open-telemetry/otel-arrow/collector/processor/obfuscationprocessor - + # We emphatically DO NOT recommend use of the memory limiter # processor or the core batch processor, i.e., do not build with # either of: @@ -115,32 +115,32 @@ processors: # - go.opentelemetry.io/collector/processor/batchprocessor. # These Collector-Contrib components are referred to in the charts in this package. - - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor v0.91.0 - - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/k8sattributesprocessor v0.91.0 - - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourceprocessor v0.91.0 - - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/metricstransformprocessor v0.91.0 + - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor v0.94.0 + - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/k8sattributesprocessor v0.94.0 + - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourceprocessor v0.94.0 + - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/metricstransformprocessor v0.94.0 # ServiceNow does not at this time recommend the probabilistic sampler processor, # because it leads to undercounting of spans. We will update this recommendation # when the sampler is fully supported. - # - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/probabilisticsamplerprocessor v0.91.0 - + # - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/probabilisticsamplerprocessor v0.94.0 + # You may be interested in other transform components in the # Collector-Contrib repository. Here are some that might be useful: # - # - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/attributesprocessor v0.91.0 + # - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/attributesprocessor v0.94.0 extensions: # You may be interested in externsions from the Collector-Contrib repository. None are # required. Here are some that might be useful: - - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/extension/healthcheckextension v0.91.0 - - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/extension/pprofextension v0.91.0 + - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/extension/healthcheckextension v0.94.0 + - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/extension/pprofextension v0.94.0 # We do not support use of headersetterextension to apply the lightstep-access-token # header on a per-request basis. Contact ServiceNow if you are interested in # multi-tenancy with OTel-Arrow. Do not build with # github.com/open-telemetry/opentelemetry-collector-contrib/extension/headerssetterextension - + # If you are building this collector in an Golang monorepo, or to apply security patches # and other overrides to the go.mod file, use this section. replaces: From 1a0db505bbbe04d39568d14847b672ffd615bfd9 Mon Sep 17 00:00:00 2001 From: Jacob Aronoff Date: Thu, 15 Feb 2024 14:48:41 -0500 Subject: [PATCH 2/2] enable processor --- arrow/otelcolarrow-build.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arrow/otelcolarrow-build.yaml b/arrow/otelcolarrow-build.yaml index 0ce442c..045d040 100644 --- a/arrow/otelcolarrow-build.yaml +++ b/arrow/otelcolarrow-build.yaml @@ -123,7 +123,7 @@ processors: # ServiceNow does not at this time recommend the probabilistic sampler processor, # because it leads to undercounting of spans. We will update this recommendation # when the sampler is fully supported. - # - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/probabilisticsamplerprocessor v0.94.0 + - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/probabilisticsamplerprocessor v0.94.0 # You may be interested in other transform components in the # Collector-Contrib repository. Here are some that might be useful: