Skip to content

Latest commit

 

History

History
355 lines (270 loc) · 13.4 KB

Upgrading.md

File metadata and controls

355 lines (270 loc) · 13.4 KB
Raw

Upgrading

Unreleased

Upgrading to v0.57.2-sumo-0

sumologic exporter: drop support for source headers

Since the exporter should not modify the data, setting the source headers is no longer supported in the sumologic exporter. This deprecation includes deprecating the source templates.

If you need to set the _source* headers in the collector, use the transform processor, for example:

processors:
  # ...
  transform:
    logs:
      - set(attributes["_sourceHost"], "source_host")

Upgrading to v0.56.0-sumo-0

sumologic exporter: drop support for translating attributes

Translating the attributes is harmless, but the exporters should not modify the data. This functionality has been moved to the sumologicschema processor. Due to that, this functionality is now deprecated in the exporter and will be removed soon.

However, if the attributes are not translated, some Sumo Logic apps might not work correctly. To migrate, add a sumologicschema processor to your pipelines that use the sumologic exporter and disable that functionality in the exporter:

processors:
  # ...
  sumologic_schema:
    translate_attributes: true

exporters:
  sumologic:
    # ...
    translate_attributes: false

# ...

service:
  pipelines:
    logs:
      processors:
        # - ...
        - sumologic_schema

Note: by default, the sumologicschema processor also performs other actions, like adding cloud.namespace attribute to the data. If you don't want this to happen, you should explicitly disable these functionalities:

processors:
  sumologic_schema:
    add_cloud_namespace: false
    # ...

Full list of configuration settings can be found in the readme of the processor.

sumologic exporter: drop support for translating Telegraf metric names

Similar as above, the translation should not happen in the exporter and has been moved to the sumologicschema processor. The functionality is now deprecated and will be removed soon.

However, if the attributes are not translated, some Sumo Logic apps might not work correctly. To migrate, add a sumologicschema processor to your pipelines that use the sumologic exporter and disable that functionality in the exporter:

processors:
  # ...
  sumologic_schema:
    translate_telegraf_attributes: true

exporters:
  sumologic:
    # ...
    translate_telegraf_attributes: false

# ...

service:
  pipelines:
    logs:
      processors:
        # - ...
        - sumologic_schema

Note: By default, the sumologicschema processor also performs other actions. Please see a corresponding warning in paragraph sumologic exporter: drop support for translating attributes for more information.

Upgrading to v0.55.0-sumo-0

filter processor: drop support for expr language

Expr language is supported by logstransform processor, so there is no need to have this functionality in filter processor.

The following configuration of filter processor:

processors:
  filter:
    logs:
      include:
        match_type: expr
        expressions:
          - Body matches "log to include"
      exclude:
        match_type: expr
        expressions:
          - Body matches "log to exclude"

is equivalent of the following configuration of logstransform processor:

processors:
  logstransform:
    operators:
      - type: filter
        expr: 'body matches "log to include"'
      - type: filter
        expr: 'not(body matches "log to exclude")'

Upgrading to v0.52.0-sumo-0

sumologic exporter: Removed carbon2 and graphite metric formats

These metric formats don't offer any advantages over Prometheus or OTLP formats. To migrate, simply switch the format to prometheus.

exporters:
  sumologic:
    metric_format: prometheus

Upgrading to v0.51.0-sumo-0

k8s_tagger processor: removed clusterName metadata extraction option

Before v0.51.0-sumo-0, you could specify clusterName as one of the options for metadata extraction:

processors:
  k8s_tagger:
    extract:
      metadata:
      - clusterName

Starting with v0.51.0-sumo-0, the clusterName option is removed. This is a result of an upstream change that removes the k8s.cluster.name metadata (link), which in turn is a result of Kubernetes libraries removing support for this deprecated and non-functional field (link).

How to upgrade

Check if you are using the extract.metadata option in your k8s_tagger processor configuration and if yes, check if it includes the clusterName entry. If it does, remove it.

For example, change the following configuration:

processors:
  k8s_tagger:
    extract:
      metadata:
      - clusterName
      - deploymentName
      - podName
      - serviceName
      - statefulSetName

to the following, removing the clusterName entry from the list:

processors:
  k8s_tagger:
    extract:
      metadata:
      - deploymentName
      - podName
      - serviceName
      - statefulSetName

sumologic exporter: metadata translation: changed the attribute that is translated to _sourceName from file.path.resolved to log.file.path_resolved

This change should have landed already in previous version v0.49.0-sumo-0. It is a result of a change in Filelog receiver, which starting with v0.49.0, changed the names of the attributes it creates when one of the configuration properties is set to true:

  • include_file_name: changed attribute name from file.name to log.file.name
  • include_file_name_resolved: changed attribute name from file.name.resolved to log.file.name_resolved
  • include_file_path: changed attribute name from file.path to log.file.path
  • include_file_path_resolved: changed attribute name from file.path.resolved to log.file.path_resolved

See documentation that describes this change.

How to upgrade

This change is technically a breaking change, but it should be transparent and require no changes in your configuration.

If you have a pipeline that includes Filelog receiver (configured with include_file_path_resolved: true) and Sumo Logic exporter, the Filelog receiver will create the log.file.path_resolved attribute and Sumo Logic exporter will translate this attribute to _sourceName.

exporters:
  sumologic:
    endpoint: ...

receivers:
  filelog:
    include:
    - ...
    include_file_path_resolved: true

service:
  pipelines:
    logs:
      exporters:
      - sumologic
      receivers:
      - filelog

In the unlikely scenario that you have a component other than Filelog receiver that creates the file.path.resolved attribute that you relied on Sumo Logic exporter to be translated to _sourceName attribute, you can perform the translation with Resource processor like the following:

processors:
  resource:
    attributes:
    - action: insert
      from_attribute: file.path.resolved
      key: _sourceName
    - action: delete
      key: file.path.resolved

Upgrading to 0.49.0-sumo-0

Several changes to receivers using opentelemetry-log-collection

The affected receivers are: filelog, syslog, tcplog, udplog, and journald.

opentelemetry-log-collection is a shared library used by the aforementioned receivers. This release contains several breaking changes to its configuration syntax. Please refer to the official upgrade guide for more information.

Sumo Logic exporter metadata handling

The OpenTelemetry data format makes a distinction between record-level attributes and resource-level attributes. The metadata_attributes configuration option in the sumologicexporter allowed setting metadata for records sent to the Sumo Logic backend based on both record and resource-level attributes. Only attributes matching the supplied regular expressions were sent.

However, this is conceptually incompatible with OpenTelemetry. Our intent with the exporter is to use OpenTelemetry conventions as much as we can, to the point where it should eventually be possible to export data to Sumo using the upstream OTLP exporter. This is why we are changing the behaviour. From now on:

  1. metadata_attributes no longer exists.
  2. Metadata for sent records is based on resource-level attributes.

In order to retain current behaviour, processors should be used to transform the data before it is exported. This potentially involves two transformations:

Removing unnecessary metadata using the resourceprocessor

metadata_attributes allowed filtering based on regular expressions. An equivalent processor doesn't yet exist, but resource-level attributed can be dropped using the resourceprocessor. For example:

processors:
  resource:
    attributes:
      - pattern: ^k8s\.pod\..*
        action: delete

will delete all attributes starting with k8s.pod..

NOTE: The ability to delete attributes based on a regular expression is currently unique to our fork of the resourceprocessor, and isn't available in upstream.

Moving record-level attributes used for metadata to the resource level

This can be done using the Group by Attributes processor. If you were using the Sumo Logic exporter to export data with a host record-level attribute:

exporters:
  sumologicexporter:
    ...
    metadata_attributes:
      - host

You can achieve the same effect with the following processor configuration:

processors:
  groupbyattrsprocessor:
    keys:
      - host

Keep in mind that your attribute may already be resource-level, in which case no changes are necessary.