Proposal for Log and Event capturing with auto instrumentation

[aabmass]

Maybe rehashing #4269 a little bit here.

Current behavior

The current behavior of auto instrumentation is that Logs and Events are NOOP by default. That means any logs created through the Events API will be dropped.

However, if you set OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true code, then we:

1. set up real global LoggerProvider, EventLoggerProvider
2. set the OTel LoggingHandler on Python's root logger

Number 2 causes python's builtin logging logs to be captured only in the logging SDK. If they were previously being printed, they no longer will be.

Proposal

By default:

• Events should be captured
• users should continue stdlib logging calls outputted as they were without auto instrumentation (see Consider moving LoggingHandler out of the SDK and into an instrumentation #4330)

We could also consider teeing logs so they are both captured in the SDK and get printed out.

[xrmx]

I'm in favor of moving the setup of LoggerProvider and EventLoggerProvider out of OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED guard.

[lzchen]

One thing of note is that users have to explicitly set what exporter is needed when using autoinstrumentation (or elses otlp/grpc will be used by default) -> opentelemetry-instrument --traces_exporter console --metrics_exporter None --logs_exporter None python main.py.

Those not interested in logging before will now have to explicitly set this (instead of OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED ). Also I've always been a little in favor of making the default behavior console but this might be too big of a breaking change for traces and metrics if we do alltogether.

[aabmass]

Those not interested in logging before will now have to explicitly set this (instead of OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED ).

I didn't follow your concern @lzchen. This proposal shouldn't have any affect on users who don't want OTel to mess with stdlib logging.

[lzchen]

If we enable setting up events/log by default (without relying on OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED, users will now need to explicitly set logs_exporter to None if they are using autoinstrumentation and NOT interested in logging (because by default, an otlp pipeline is setup for each signal if nothing is set for those parameters) otherwise they would get erroneous collector endpoint missing exceptions. Before when users don't have the env var set, it defaults to not setting up any logging pipeline at all.

Unless if I am misunderstanding @xrmx 's message.

[aabmass]

If we enable setting up events/log by default (without relying on OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED, users will now need to explicitly set logs_exporter to None if they are using autoinstrumentation and NOT interested in logging (because by default, an otlp pipeline is setup for each signal if nothing is set for those parameters) otherwise they would get erroneous collector endpoint missing exceptions.

What do you mean by not interesting in logging here?

The only thing that would be sending logs would be Event API instrumentation. In which case, any collector endpoint missing issues are not really erroneous

[lzchen]

What do you mean by not interesting in logging here?

If I want to use auto instrumentation but I don't want logging/metrics telemetry, perhaps due to cost reasons.

[xrmx]

One thing of note is that users have to explicitly set what exporter is needed when using autoinstrumentation (or elses otlp/grpc will be used by default) -> opentelemetry-instrument --traces_exporter console --metrics_exporter None --logs_exporter None python main.py.

I think this may already be the case since we changed the default for OTEL_LOGS_EXPORTER to otlp in the distro.

[lzchen]

I think this may already be the case since we changed the default for OTEL_LOGS_EXPORTER to otlp in the distro.

I don't think this will happen if users don't explicitly set OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED

[xrmx]

I think this may already be the case since we changed the default for OTEL_LOGS_EXPORTER to otlp in the distro.

I don't think this will happen if users don't explicitly set OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED

I think I hit this by chance yesterday when I tested a git checkout and I didn't have that environment variable set. AFAIR opentelemetry-instrument looks at the env vars set and doesn't care if things are enabled or not.

[xrmx]

Tested this again and confirm behavior already changed:

OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=false opentelemetry-instrument --traces_exporter=none --metrics_exporter=none python3 app.py
...
    raise RuntimeError(
RuntimeError: Requested component 'otlp_proto_grpc' not found in entry point 'opentelemetry_logs_exporter'