Repository navigation
fix(boto3): Trace the complete botocore client-call lifecycle - #7538
Merged
pabloDeputter merged 9 commits intoSep 30, 2026
Merged
pabloDeputter merged 9 commits into
pabloDeputter merged 9 commits into
Conversation
pabloDeputter
added this pull request to stack #7539
September 18, 2026 14:23
Contributor
Codecov Results 📊✅ 132487 passed | ⏭️ 7220 skipped | Total: 139707 | Pass Rate: 94.83% | Execution Time: 454m 48s 📊 Comparison with Base Branch
➖ Removed Tests (60)View removed tests
All tests are passing successfully. ✅ Patch coverage is 95.51%. Project has 2551 uncovered lines. Files with missing lines (3)
Coverage diff@@ Coverage Diff @@
## master #PR +/-##
==========================================
+ Coverage 90.31% 90.35% +0.04%
==========================================
Files 195 199 +4
Lines 26261 26422 +161
Branches 9798 9834 +36
==========================================
+ Hits 23714 23871 +157
- Misses 2547 2551 +4
- Partials 1485 1491 +6Generated by Codecov Action |
pabloDeputter
force-pushed
the
pablo/trace-boto3-client-call-lifecycle
branch
from
September 18, 2026 15:06
53a8717 to
4efce4b
Compare
pabloDeputter
removed this pull request from stack #7539
September 18, 2026 15:08
pabloDeputter
changed the base branch from
pablo/refactor-boto3-integration
to
pablo/harden-boto3-streaming-body
September 18, 2026 15:08
pabloDeputter
added this pull request to stack #7541
September 18, 2026 15:08
pabloDeputter
force-pushed
the
pablo/trace-boto3-client-call-lifecycle
branch
from
September 18, 2026 15:15
4efce4b to
05501d2
Compare
pabloDeputter
marked this pull request as ready for review
September 18, 2026 15:15
![sentry-warden[bot]](https://avatars.githubusercontent.com/in/2755838?s=60&v=4){kind=small}
ericapisani
reviewed
Sep 18, 2026
ericapisani
left a comment
ericapisani
left a comment
Member
There was a problem hiding this comment.
Minor things but overall looking good. Will take a look at tests on my 2nd pass
| from sentry_sdk.integrations.boto3 import Boto3Integration | ||
| @contextmanager | ||
| def _activate_client_span(span: "StreamedSpan") -> "Iterator[StreamedSpan]": | ||
| """Temporarily activate an inactive boto span without ending it.""" |
Member
There was a problem hiding this comment.
Worth adding context on why we're doing this.
| # use unknown if `service_id_hyphenized` so span name can still be created. | ||
| # e.g. "aws.unkown.GetObject" | ||
| service_name = ctx.service_id_hyphenized or "unknown" | ||
| span_name = "aws.%s.%s" % (service_name, ctx.operation_name) |
Member
There was a problem hiding this comment.
f-strings are generally the preferred way to construct strings like this in modern python these days
Suggested change
| span_name = "aws.%s.%s" % (service_name, ctx.operation_name) | |
| span_name = f"aws.{service_name}.{ctx.operation_name}" |
Member
Author
There was a problem hiding this comment.
yupp, I took over the old code, but I usually also use f-strings.
| SPANDATA.SENTRY_ORIGIN: span_origin, | ||
| } | ||
| if ctx.service_id: | ||
| attributes[SPANDATA.RPC_METHOD] = "%s/%s" % ( |
| "sentry.op": OP.HTTP_CLIENT_STREAM, | ||
| "sentry.origin": Boto3Integration.origin, | ||
| SPANDATA.SENTRY_OP: OP.HTTP_CLIENT_STREAM, | ||
| SPANDATA.SENTRY_ORIGIN: Boto3Integration.origin, |
Comment on lines
+279
to
+281
| if isinstance(span, StreamedSpan) and ( | ||
| span.get_attributes().get(SPANDATA.SENTRY_ORIGIN) != Boto3Integration.origin | ||
| ): |
Member
There was a problem hiding this comment.
Because of the length of this conditional, I think it'd be a bit cleaner to pull this into a variable
Suggested change
| if isinstance(span, StreamedSpan) and ( | |
| span.get_attributes().get(SPANDATA.SENTRY_ORIGIN) != Boto3Integration.origin | |
| ): | |
| is_span_origin_from_boto = span.get_attributes().get(SPANDATA.SENTRY_ORIGIN) == Boto3Integration.origin | |
| if isinstance(span, StreamedSpan) and is_span_origin_from_boto: |
Member
Author
There was a problem hiding this comment.
I agree, but then you get the issue that the legacy span doesn't support get_attributes(); I fixed it like this:
# an ignored streamed span is not activated; avoid enriching its parent.
if isinstance(span, StreamedSpan):
if not (span.get_attributes().get(SPANDATA.SENTRY_ORIGIN) == ORIGIN):
return| }, | ||
| # boto3 integration owns span's lifecycle; keep child inactive so it | ||
| # can't restore boto3 span later on. | ||
| active=not is_inactive_boto3_span, |
Member
There was a problem hiding this comment.
Apologies for this slight nitpick, but can we add a space after the = and before the not?
The "not" almost blends in with the =, especially because the syntax highlighting of Github and IDEs use the same colour for the two. 😭
Member
Author
There was a problem hiding this comment.
I agree, but the linter doesn't really like that 🤣 I fixed it by adding fmt: off and fmt: on for that specific block; I haven't seen it being used anywhere else in the codebase, so not sure whether this is correct.
pabloDeputter
force-pushed
the
pablo/trace-boto3-client-call-lifecycle
branch
from
September 21, 2026 09:19
05501d2 to
421da22
Compare
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
pabloDeputter
force-pushed
the
pablo/trace-boto3-client-call-lifecycle
branch
from
September 21, 2026 09:38
c3e8f5b to
66d4665
Compare
pabloDeputter
force-pushed
the
pablo/trace-boto3-client-call-lifecycle
branch
2 times, most recently
from
September 21, 2026 10:54
fe64ae9 to
add0913
Compare
pabloDeputter
force-pushed
the
pablo/trace-boto3-client-call-lifecycle
branch
3 times, most recently
from
September 22, 2026 08:52
f649477 to
4423557
Compare
pabloDeputter
force-pushed
the
pablo/trace-boto3-client-call-lifecycle
branch
from
September 22, 2026 09:18
4423557 to
c622458
Compare
pabloDeputter
force-pushed
the
pablo/trace-boto3-client-call-lifecycle
branch
from
September 22, 2026 14:53
c622458 to
3e4fcfd
Compare
pabloDeputter
removed this pull request from stack #7541
September 22, 2026 14:57
pabloDeputter
added this pull request to stack #7618
September 22, 2026 14:57
ericapisani
approved these changes
Sep 28, 2026
ericapisani
left a comment
ericapisani
left a comment
Member
There was a problem hiding this comment.
Some minor things but otherwise LGTM 🚀 Approving to not block
| Track a single API call, including retries, serialization, and endpoint | ||
| resolution. For streaming responses, keep the span open until the | ||
| response body is consumed or closed. | ||
| https://github.com/boto/botocore/blob/develop/botocore/client.py |
Member
There was a problem hiding this comment.
I think something got borked in a merge conflict resolution - I can see you made the change in this commit but for some reason it's still using the old path 🤔
Member
Author
There was a problem hiding this comment.
yeah I vaguely remember fixing it; so don't know why it was removed
| origin=ORIGIN, | ||
| return None | ||
|
|
||
| # use unknown if `service_id_hyphenized` so span name can still be created. |
Member
There was a problem hiding this comment.
This looks like a partially incomplete comment - is it supposed to instead be the following?
Suggested change
| # use unknown if `service_id_hyphenized` so span name can still be created. | |
| # use unknown if `service_id_hyphenized` is not set so span name can still be created. |
| return None | ||
|
|
||
| # use unknown if `service_id_hyphenized` so span name can still be created. | ||
| # e.g. "aws.unkown.GetObject" |
Member
There was a problem hiding this comment.
Suggested change
| # e.g. "aws.unkown.GetObject" | |
| # e.g. "aws.unknown.GetObject" |
pabloDeputter
force-pushed
the
pablo/trace-boto3-client-call-lifecycle
branch
from
September 29, 2026 08:34
5b894cb to
440b19a
Compare
pabloDeputter
force-pushed
the
pablo/trace-boto3-client-call-lifecycle
branch
from
September 30, 2026 12:04
440b19a to
04d968a
Compare
pabloDeputter
removed this pull request from stack #7618
September 30, 2026 12:05
pabloDeputter
added this pull request to stack #7792
September 30, 2026 12:05
pabloDeputter
force-pushed
the
pablo/trace-boto3-client-call-lifecycle
branch
from
September 30, 2026 13:31
04d968a to
33807b8
Compare
Comment on lines
+171
to
+174
| assert len(http_spans) == 1 | ||
| assert len(stream_spans) == 1 | ||
| client_span = client_spans[0] | ||
| http_span = http_spans[0] |
Contributor
There was a problem hiding this comment.
AWS connection putrequest wrapper bypasses the stdlib span wrapper
_install_httplib wraps AWSHTTPConnection.putrequest before replacing HTTPConnection.putrequest, so the AWS wrapper captures the unpatched inherited method. The AWS class then shadows the patched base method, and boto HTTP requests skip stdlib HTTP span setup, including setting _sentrysdk_real_url (the boto client span is still created). Install the base wrapper before wrapping the AWS classes.
Evidence
_install_httplibsaves the originalHTTPConnection.putrequestand defines the sentry wrapper that creates the HTTP span and sets_sentrysdk_real_url.- It calls
_patch_aws_connection()before assigning that wrapper toHTTPConnection.putrequest. _patch_aws_connection()wrapsAWSHTTPConnection.putrequestat that point, capturing the unpatched method; the AWS class method then shadows the later base-class patch, so requests through it skip sentryputrequest.
Identified by Warden · code-review, find-bugs · GUM-LY4
pabloDeputter
force-pushed
the
pablo/trace-boto3-client-call-lifecycle
branch
from
September 30, 2026 14:05
33807b8 to
2647da1
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 2647da1. Configure here.
6 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Move boto3 span creation from individual HTTP request attempts to full botocore client-call lifecycle.
Previously, the client span was created from the botocore
request-createdevent; since botocore creates a newAwsRequestevent for every retry, these spans represent individual retries rather than the full event. Following OTel (https://opentelemetry.io/docs/specs/semconv/rpc/rpc-spans/#rpc-client-span) the span should cover the entire call lifecycle including all retries. Wrapping_make_api_call()covers all retries performed by botocore, including serialization, endpoint resolution, the final response or failures (https://github.com/boto/botocore/blob/develop/botocore/client.py:999).BaseClient._make_api_call()so one span represents a single boto operation; this span is also kept active across every retry attempt.request-createdis kept for breadcrumbs, HTTP request attributes, and trace propagation.StreamingBodyare kept open until body is consumed or closed.Issues
Resolves #7474