Skip to content

Introduce new exporter helper with batching option #8122

Open
7 of 10 issues completed
Open
7 of 10 issues completed
@dmitryax

Description

@dmitryax

This is a tracking issue for introducing the new exporter helper and migrating the existing exporters to use it.

The primary reason for introducing the new exporter helper is to move the batching to the exporter side and deprecate the batch processor as part of making the delivery pipeline reliable, as reported in #7460. More details about moving batching to the exporter helper can be found in #4646.

Shifting batching to the exporter side grants us the opportunity to leverage the exporter's data model instead of relying on OTLP. As a result, we can achieve the following benefits:

  • Ability to place failed requests back into the queue without the need for converting them back to OTLP format.
  • Enhanced control in counting queue and batch sizes using basic items (like spans, data points, or log records for OTLP) tailored to different exporters, resolving the concern raised in issue Processor: Splitting summary metrics into timeseries. opentelemetry-collector-contrib#7134.
  • Optional counting of queue and batch sizes in bytes of serialized data.

Adapting to the new exporter helper requires exporter developers to implement at least two functions:

  1. Converter: to translate pdata Metrics/Traces/Logs into a user-defined Request.
  2. Request sender: to send the user-defined Request

Design document: https://docs.google.com/document/d/1uxnn5rMHhCBLP1s8K0Pg_1mAs4gCeny8OWaYvWcuibs

(edit by @mx-psi) For essential sub-issues to mark this as complete see the 'Sub-issues' list below.

Additional sub-issues to get feature parity with the batch processor:

Sub-issues

Metadata

Metadata

Assignees

Type

No type

Projects

Status

In Progress

Relationships

None yet

Development

No branches or pull requests

Issue actions