fix(boto3): Trace the complete botocore client-call lifecycle by pabloDeputter · Pull Request #7538 · getsentry/sentry-python

pabloDeputter · 2026-09-18T14:23:33Z

Description

Move boto3 span creation from individual HTTP request attempts to full botocore client-call lifecycle.

Previously, the client span was created from the botocore request-created event; since botocore creates a new AwsRequest event for every retry, these spans represent individual retries rather than the full event. Following OTel (https://opentelemetry.io/docs/specs/semconv/rpc/rpc-spans/#rpc-client-span) the span should cover the entire call lifecycle including all retries. Wrapping _make_api_call() covers all retries performed by botocore, including serialization, endpoint resolution, the final response or failures (https://github.com/boto/botocore/blob/develop/botocore/client.py:999).

patch BaseClient._make_api_call() so one span represents a single boto operation; this span is also kept active across every retry attempt.
request-created is kept for breadcrumbs, HTTP request attributes, and trace propagation.
client spans that fail will preserve the original exception.
responses that return a StreamingBody are kept open until body is consumed or closed.
fix the correct parent/child relationship between boto, HTTP, and streaming spans.

Issues

Resolves #7474

github-actions · 2026-09-18T14:34:39Z

Codecov Results 📊

✅ 129901 passed | ⏭️ 7171 skipped | Total: 137072 | Pass Rate: 94.77% | Execution Time: 454m 54s

📊 Comparison with Base Branch

Metric	Change
Total Tests	📈 +546
Passed Tests	📈 +523
Failed Tests	—
Skipped Tests	📈 +23

All tests are passing successfully.

✅ Patch coverage is 95.51%. Project has 2546 uncovered lines.
✅ Project coverage is 90.27%. Comparing base (ff199b0) to head (f649477).

Files with missing lines (3)

File	Patch %	Lines
sentry_sdk/integrations/boto3/_instrumentation.py	94.51%	⚠️ 5 Missing and 11 partials
sentry_sdk/integrations/boto3/_client.py	96.23%	⚠️ 2 Missing and 1 partials
sentry_sdk/integrations/boto3/_context.py	96.30%	⚠️ 1 Missing and 2 partials

Coverage diff

@@            Coverage Diff             @@
##        master       #PR       +/-##
==========================================
+ Coverage    90.21%    90.27%    +0.06%
==========================================
  Files          194       198        +4
  Lines        25888     26156      +268
  Branches      9584      9696      +112
==========================================
+ Hits         23352     23610      +258
- Misses        2536      2546       +10
- Partials      1455      1456        +1

Generated by Codecov Action

ericapisani

Minor things but overall looking good. Will take a look at tests on my 2nd pass

ericapisani · 2026-09-18T19:37:31Z

-    from sentry_sdk.integrations.boto3 import Boto3Integration
+@contextmanager
+def _activate_client_span(span: "StreamedSpan") -> "Iterator[StreamedSpan]":
+    """Temporarily activate an inactive boto span without ending it."""


Worth adding context on why we're doing this.

ericapisani · 2026-09-18T19:55:02Z

+    # use unknown if `service_id_hyphenized` so span name can still be created.
+    # e.g. "aws.unkown.GetObject"
+    service_name = ctx.service_id_hyphenized or "unknown"
+    span_name = "aws.%s.%s" % (service_name, ctx.operation_name)


f-strings are generally the preferred way to construct strings like this in modern python these days

Suggested change

span_name = "aws.%s.%s" % (service_name, ctx.operation_name)

span_name = f"aws.{service_name}.{ctx.operation_name}"

yupp, I took over the old code, but I usually also use f-strings.

ericapisani · 2026-09-18T19:55:27Z

+            SPANDATA.SENTRY_ORIGIN: span_origin,
+        }
+        if ctx.service_id:
+            attributes[SPANDATA.RPC_METHOD] = "%s/%s" % (


Same f-string comment here

ericapisani · 2026-09-18T19:55:58Z

-                "sentry.op": OP.HTTP_CLIENT_STREAM,
-                "sentry.origin": Boto3Integration.origin,
+                SPANDATA.SENTRY_OP: OP.HTTP_CLIENT_STREAM,
+                SPANDATA.SENTRY_ORIGIN: Boto3Integration.origin,


Nice change👌🏻

ericapisani · 2026-09-18T19:58:58Z

+        if isinstance(span, StreamedSpan) and (
+            span.get_attributes().get(SPANDATA.SENTRY_ORIGIN) != Boto3Integration.origin
+        ):


Because of the length of this conditional, I think it'd be a bit cleaner to pull this into a variable

Suggested change

if isinstance(span, StreamedSpan) and (

span.get_attributes().get(SPANDATA.SENTRY_ORIGIN) != Boto3Integration.origin

):

is_span_origin_from_boto = span.get_attributes().get(SPANDATA.SENTRY_ORIGIN) == Boto3Integration.origin

if isinstance(span, StreamedSpan) and is_span_origin_from_boto:

I agree, but then you get the issue that the legacy span doesn't support get_attributes(); I fixed it like this:

# an ignored streamed span is not activated; avoid enriching its parent. if isinstance(span, StreamedSpan): if not (span.get_attributes().get(SPANDATA.SENTRY_ORIGIN) == ORIGIN): return

ericapisani · 2026-09-18T20:00:48Z

+                headers["foo"] = "old"
+                headers["foo"] = "new"
+            produces two fields: {"foo": "old", "foo": "new"}. So delete existing
+            fields before assigning replacement.


Great comment, thanks for adding this 👍🏻 🙏🏻

yeah it's really weird behavior 😆

ericapisani · 2026-09-18T20:06:00Z

                    },
+                    # boto3 integration owns span's lifecycle; keep child inactive so it
+                    # can't restore boto3 span later on.
+                    active=not is_inactive_boto3_span,


Apologies for this slight nitpick, but can we add a space after the = and before the not?

The "not" almost blends in with the =, especially because the syntax highlighting of Github and IDEs use the same colour for the two. 😭

I agree, but the linter doesn't really like that 🤣 I fixed it by adding fmt: off and fmt: on for that specific block; I haven't seen it being used anywhere else in the codebase, so not sure whether this is correct.

cursor

Stale Bugbot comment from a previous run.

cursor

Stale Bugbot comment from a previous run.

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 177d277. Configure here.}

pabloDeputter added this pull request to stack #7539 September 18, 2026 14:23

pabloDeputter force-pushed the pablo/trace-boto3-client-call-lifecycle branch from 53a8717 to 4efce4b Compare September 18, 2026 15:06

pabloDeputter removed this pull request from stack #7539 September 18, 2026 15:08

pabloDeputter changed the base branch from pablo/refactor-boto3-integration to pablo/harden-boto3-streaming-body September 18, 2026 15:08

pabloDeputter added this pull request to stack #7541 September 18, 2026 15:08

pabloDeputter force-pushed the pablo/trace-boto3-client-call-lifecycle branch from 4efce4b to 05501d2 Compare September 18, 2026 15:15

pabloDeputter marked this pull request as ready for review September 18, 2026 15:15

pabloDeputter requested a review from a team as a code owner September 18, 2026 15:15

sentry-warden Bot reviewed Sep 18, 2026

View reviewed changes

Comment thread sentry_sdk/integrations/boto3/_client.py Outdated

sentry-warden Bot reviewed Sep 18, 2026

View reviewed changes

Comment thread tests/integrations/boto3/test_client.py Outdated

ericapisani reviewed Sep 18, 2026

View reviewed changes

pabloDeputter force-pushed the pablo/trace-boto3-client-call-lifecycle branch from 05501d2 to 421da22 Compare September 21, 2026 09:19

cursor Bot reviewed Sep 21, 2026

View reviewed changes

pabloDeputter closed this Sep 21, 2026

pabloDeputter force-pushed the pablo/trace-boto3-client-call-lifecycle branch from c3e8f5b to 66d4665 Compare September 21, 2026 09:38

pabloDeputter reopened this Sep 21, 2026

cursor Bot reviewed Sep 21, 2026

View reviewed changes

Comment thread sentry_sdk/integrations/boto3/_instrumentation.py

pabloDeputter force-pushed the pablo/trace-boto3-client-call-lifecycle branch 2 times, most recently from fe64ae9 to add0913 Compare September 21, 2026 10:54

cursor Bot reviewed Sep 21, 2026

View reviewed changes

Comment thread sentry_sdk/integrations/boto3/_client.py

pabloDeputter force-pushed the pablo/trace-boto3-client-call-lifecycle branch from 9091f42 to d473247 Compare September 21, 2026 15:39

pabloDeputter added 6 commits September 21, 2026 18:07

fix(boto3): trace the complete client-call lifecycle

f977027

fix merging issues

983ad0e

review changes

329d5a3

add test for when StreamingBody instrumentation setup fails

58826f1

fix(boto3): finish streaming spans before legacy boto spans

40f47b0

add permalink

02c00cc

add some more comments

f649477

pabloDeputter force-pushed the pablo/trace-boto3-client-call-lifecycle branch from d473247 to f649477 Compare September 21, 2026 16:07

	span_name = "aws.%s.%s" % (service_name, ctx.operation_name)
	span_name = f"aws.{service_name}.{ctx.operation_name}"

Conversation

pabloDeputter commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Issues

Uh oh!

github-actions Bot commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Results 📊

📊 Comparison with Base Branch

Uh oh!

Uh oh!

Uh oh!

ericapisani left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pabloDeputter Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pabloDeputter commented Sep 18, 2026 •

edited

Loading

github-actions Bot commented Sep 18, 2026 •

edited

Loading

pabloDeputter Sep 21, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading