New component: Enrich Attributes based on key matching from yaml or csv definition

[sgerme-cpe-obs-dkt]

The purpose and use-cases of the new component

We have developed an internal component, enrichprocessor, which we believe could be highly valuable to the community. It was designed to address several use cases:

• Attribute Mapping: Enrich signals by mapping a technical identifier (UUID) to a business-friendly name, based on a correspondence file (CSV).

• Value Replacement: Replace coded values (e.g., an integer in a log representing an HTTP method) with their human-readable equivalents.

• Data Enrichment: Populate missing or incorrect attributes, such as incomplete Kubernetes metadata or other missing information.

• Attribute Augmentation for Routing: Modify attributes to facilitate signal routing. For example, by appending new values to an existing attribute, we enable routing decisions (e.g., Datadog, Splunk) based on enriched criteria, typically processed later by a routing connector.

Example configuration for the component

  enrich:
    jobs:
      - key: "prod"
        match_attr_key: "env"
        is_for_resource: true
        values:
          - key: "region"
            val: "eu"
            replace: false

processors:
  enrich:
    source_type: csv
    jobs_file:
      path: "./testdata/enrich.csv"
      replace: false
      is_for_resource: true

with

env,team,region
prod,observability,eu
stage,platform,us-central1

Telemetry data types supported

traces, metrics, logs

Code Owner(s)

No response

Sponsor (optional)

No response

Additional context

code is already written and tested.

[axw]

The general capability sounds sensible, much like Logstash's translate filter: https://www.elastic.co/docs/reference/logstash/plugins/plugins-filters-translate

IIUC, some of these things can already be done by the transform processor. The missing capabilities seem to be:

• loading data from an external source, e.g. a file
• OTTL editors (functions) for dynamically setting attributes, like in your CSV example

The new function could probably be implemented independently of externally sourced data, or its original format.

It may also be useful to load data from other sources, e.g. from a SQL database, or an HTTP API. Maybe not for your use case, but see https://www.elastic.co/docs/reference/logstash/lookup-enrichment. So what I'm wondering is whether we should introduce some kind of "lookup extension" API which is then implemented by a (CSV/YAML/JSON) file-based extension. The transform processor would then need to be taught how to load such extensions and make them available to OTTL statements.

Anyway, just my 2c to make this compose with existing functionality.

If you are intending to contribute your existing code, then I assume you would be the code owner? If so, please update the "Code owners" section of the description. You will also need to find an approver or maintainer to sponsor the new component. Raising this issue in the CNCF Slack and/or attending a Collector SIG meeting may help - see https://github.com/open-telemetry/opentelemetry-collector?tab=readme-ov-file#community

[axw]

Related to what I'm suggesting above: #39296