Skip to content

feat(api_core): add HTTP transport attempt logging in _TraceContext - #19011

Draft
chalmerlowe wants to merge 1 commit into
feat/otel-logging-m3-s3-grpc-attempt-loggingfrom
feat/otel-logging-m3-s4-http-attempt-logging
Draft

chalmerlowe wants to merge 1 commit into
feat/otel-logging-m3-s3-grpc-attempt-loggingfrom
feat/otel-logging-m3-s4-http-attempt-logging

Conversation

@chalmerlowe

Copy link
Copy Markdown
Contributor

Problem / Context

To capture failed HTTP wire attempts (status codes >= 400 and transport dispatch exceptions) across synchronous and asynchronous REST transports — including when OpenTelemetry tracing is disabled (self._span is None) — _TraceContext in google.api_core._observability needs to invoke log_transport_attempt from record_response, record_error, and __exit__.

Bug: b/479827845

Solution

  • Add _TraceContext._log_http_attempt in google/api_core/_observability.py to emit fail-open DEBUG transport attempt logs via log_transport_attempt.
  • Update _TraceContext.record_response to log HTTP error responses (code_int >= 400) before checking self._span, ensuring HTTP attempt logs are emitted even when OpenTelemetry tracing is disabled.
  • Update _TraceContext.record_error and _TraceContext.__exit__ to log unhandled HTTP dispatch exceptions whether or not an active OpenTelemetry span is present.
  • Add test_trace_context_http_logging_on_error_status_and_exception in tests/unit/test_observability.py.

Verification

Relying on GitHub Actions CI for full matrix verification, alongside local unit test (unit-3.14) and lint verification.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces HTTP attempt logging for failed requests and dispatch exceptions within _TraceContext by adding the _log_http_attempt helper and updating record_response, record_error, and __exit__ methods, alongside adding corresponding unit tests. The review feedback suggests improving _log_http_attempt by logging exceptions instead of silencing them, avoiding defensive getattr checks for attributes initialized in __init__, and preventing duplicate logs.

Comment on lines +1390 to +1414
try:
resolved_url = (
getattr(self._request, "url", None)
if self._request is not None
else self._url
)
resolved_method = (
getattr(self._request, "method", None)
if self._request is not None
else self._method
)
log_transport_attempt(
span=self._span,
target=resolved_url,
status_code=status_code,
kind="http",
http_method=resolved_method,
exc=exc,
client_options=self._client_options,
status_message=status_message,
)
# Intentionally catch broad Exception so telemetry logging fails open
# and never disrupts the caller's RPC flow.
except Exception:
pass

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

There are two potential issues in _log_http_attempt:

  1. Double-logging / Duplicate attempts: Tracking the attempt status is a good approach to prevent duplicate logs.
  2. Silent failures: The use of a broad except Exception: pass block masks potential issues. Please log the exception using logger.warning instead of silencing it. Additionally, avoid using getattr for attributes that should be initialized in __init__ to keep the code clean and explicit.
        if getattr(self, "_attempt_logged", False):
            return
        self._attempt_logged = True
        try:
            request = self._request
            resolved_url = getattr(request, "url", None) if request is not None else self._url
            resolved_method = getattr(request, "method", None) if request is not None else self._method
            log_transport_attempt(
                span=self._span,
                target=resolved_url,
                status_code=status_code,
                kind="http",
                http_method=resolved_method,
                exc=exc,
                client_options=self._client_options,
                status_message=status_message,
            )
        except Exception as e:
            logger.warning("Telemetry logging failed: %s", e)
References
  1. Do not use defensive getattr checks for attributes that are guaranteed to be initialized in the class's init method.
  2. Avoid broad except Exception blocks that silently return None. Instead, log the exception to aid in debugging.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review!

  1. In GAPIC REST transports (_shared_macros.j2), trace_http_request only wraps the raw session dispatch and record_response(response), while HTTP status exceptions are raised outside the with block after it exits. As a result, record_response and __exit__ (record_error) are mutually exclusive in transport execution and a deduplication flag is not required here.
  2. except Exception: pass is intentional and matches the fail-open observability contract across _observability.py (log_transport_attempt, record_response, record_error, and the gRPC interceptors) so internal telemetry failures never emit unexpected warnings into user logs.
  3. self._request, self._url, and self._method are accessed directly on self; getattr is only used on the external self._request object because caller-supplied request objects are not guaranteed to define .url or .method.

@chalmerlowe
chalmerlowe force-pushed the feat/otel-logging-m3-s3-grpc-attempt-logging branch from 0b90e49 to 42a6423 Compare November 6, 2026 16:17
@chalmerlowe
chalmerlowe force-pushed the feat/otel-logging-m3-s4-http-attempt-logging branch from fd80c07 to cfdc057 Compare November 6, 2026 16:17

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant