Skip to content

fix(telemetry): raise stdout StreamHandlers to WARN when LoggingHandler is installed - #347

Closed
jeanscherf wants to merge 2 commits into
mainfrom
fix/logging-handler-info-floor
Closed

jeanscherf wants to merge 2 commits into
mainfrom
fix/logging-handler-info-floor

Conversation

@jeanscherf

@jeanscherf jeanscherf commented Sep 21, 2026 •

Copy link
Copy Markdown
Member

Summary

  • When both a StreamHandler and an OTel LoggingHandler are active on the root logger at INFO, every log record hits stdout twice: once via the StreamHandler and once via any ConsoleLogExporter in the OTel pipeline
  • The filelog receiver on the OTel collector picks up both copies, doubling CLS queue pressure and contributing to collector memory spikes on high-volume workspaces
  • After installing the LoggingHandler, raise any existing StreamHandlers on the root logger to WARN so INFO+ flows exclusively via OTLP with full trace correlation
  • Stdout still carries warnings and errors for kubectl logs visibility
  • FileHandler and other StreamHandler subclasses are not affected (exact type check)
  • Also keeps INFO floor on the LoggingHandler itself to prevent DEBUG records reaching production OTLP pipelines regardless of root logger level

Test plan

  • TestRaiseStreamHandlersToWarn::test_raises_stream_handler_below_warn — StreamHandler at DEBUG is raised to WARN
  • TestRaiseStreamHandlersToWarn::test_does_not_lower_stream_handler_above_warn — StreamHandler already at ERROR is not lowered
  • TestRaiseStreamHandlersToWarn::test_ignores_file_handler_subclass — FileHandler is not touched
  • TestMakeLoggingHandler::test_handler_level_is_info — LoggingHandler level is INFO
  • All 6 tests pass locally: uv run pytest tests/core/unit/telemetry/test_provider.py::TestMakeLoggingHandler tests/core/unit/telemetry/test_provider.py::TestRaiseStreamHandlersToWarn -v

…ords in production

LoggingHandler was created with no level (NOTSET), so if an application set
the root logger to DEBUG, every framework debug trace — httpx connection
events, OTel SDK internals, etc. — flowed into the production OTLP pipeline.

Set level=logging.INFO on the handler so the SDK always enforces a minimum
severity regardless of the application's root logger configuration.
…er is installed

When both a StreamHandler and an OTel LoggingHandler are active on the root
logger at INFO, every log record hits stdout twice — once via the StreamHandler
and once via any ConsoleLogExporter in the OTel pipeline. The filelog receiver
on the OTel collector picks up both copies, doubling CLS queue pressure and
contributing to collector memory spikes.

After installing the LoggingHandler, raise any existing StreamHandlers on the
root logger to WARN so INFO+ flows exclusively via OTLP with full trace
correlation. Stdout still carries warnings and errors for kubectl logs
visibility. FileHandler and other StreamHandler subclasses are not affected.

Also keeps INFO floor on the LoggingHandler itself to prevent DEBUG records
from reaching production OTLP pipelines regardless of root logger level.
@jeanscherf jeanscherf changed the title fix(telemetry): set INFO floor on LoggingHandler to prevent DEBUG records in production fix(telemetry): raise stdout StreamHandlers to WARN when LoggingHandler is installed Sep 21, 2026
@jeanscherf

Copy link
Copy Markdown
Member Author

Superseded by a cleaner approach — the INFO floor on the LoggingHandler was wrong as it blocks developer DEBUG logs from reaching CLS. Reopening as a focused fix that only raises stdout StreamHandlers to WARN to prevent duplication.

@jeanscherf jeanscherf closed this Sep 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant