Repository navigation
Next Python SDK major - #5005
sentrivana wants to merge 483 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #5005 +/- ##
===========================================
+ Coverage 70.55% 83.76% +13.21%
===========================================
Files 180 180
Lines 18077 18080 +3
Branches 3008 3009 +1
===========================================
+ Hits 12754 15145 +2391
+ Misses 4432 1943 -2489
- Partials 891 992 +101
|
Coverage Results 📊✅ 58898 passed | ⏭️ 3091 skipped | Total: 61989 | Pass Rate: 95.01% | Execution Time: 159m 12s 📊 Comparison with Base Branch
➖ Removed Tests (1)View removed tests
All tests are passing successfully. ✅ Patch coverage is 87.88% (3176 of 3614 changed executable lines covered; target 80%). Changed files with executable lines (146)
Coverage diff@@ Coverage Diff @@
## master #5005 +/-##
==========================================
- Coverage 90.22% 90.12% -0.10%
==========================================
Files 203 200 -3
Tracked lines 27335 22086 -5249
Branches 10246 7738 -2508
==========================================
- Hits 24661 19903 -4758
+ Misses 2674 2183 -491
+ Partials 1558 1276 -282Generated by Coverage Action |
Semver Impact of This PR⚪ None (no version bump detected) 📋 Changelog PreviewThis is how your changes will appear in the changelog. New Features ✨
Bug Fixes 🐛Anthropic
Documentation 📚
Internal Changes 🔧
Other
🤖 This preview updates automatically when you update the PR. |
Fixes for things that the bots [surfaced](#5005) on the major branch: - some version checks were too late (after patching) - fix TrytondWSGI integration name/`_MIN_VERSIONS` entry mismatch Also, changed the warning of the `DidNotEnable` message from "X not installed" to "X not installed or incompatible".
Originally raised by a bot [here](#5005 (comment)): the `parse_version` function parses version strings as is (e.g. 3.1 becomes `(3, 1)`). We use these parsed version tuples in integrations to compare the installed version against the minimum (defined in `integrations/__init__.py`). The minimum versions are often three-part, e.g. `(3, 1, 0)`. This means that we can mistakenly consider a valid version to be below the minimum, because in pure tuple terms, `(3, 1) < (3, 1, 0)` is true. This can also happen in reverse (package version has three parts, while our min version boundary has two). In this PR, we make the internal version comparison work as expected regardless of mismatches in the length of the version strings/tuples.
### Description The [OTLP integration spec](https://develop.sentry.dev/sdk/telemetry/traces/otlp/#integration-spec) says the integration **MUST NOT** set up an automatic propagator. Cross-service propagation should come from the user's OpenTelemetry propagators instead. Right now `OTLPIntegration` replaces the global OTel textmap with `SentryOTLPPropagator` by default. That means outgoing requests carry only `sentry-trace`/`baggage` and drop the W3C `traceparent`. This follows the sentry-ruby approach (getsentry/sentry-ruby#3063): - `setup_propagator` now defaults to `False`, so by default `OTLPIntegration` leaves the global textmap alone. Users can still opt in with `setup_propagator=True`. - Tests cover both the default (global textmap untouched) and the opt-in. The inject/extract tests opt in explicitly. - Added a note to the migration guide. It covers the new default, how to get the old behavior back, and how to send both W3C and Sentry headers with a `CompositePropagator`. Without any setup, OTel's default propagators apply (`tracecontext` + `baggage`, or whatever `OTEL_PROPAGATORS` says). Unlike the Ruby PR, this doesn't change the default of `setup_otlp_traces_exporter`. The spec says it defaults to `True`. Docs (`sentry-docs` OTLP page) still need to be updated to describe the new default and manual propagator setup. That isn't part of this PR. #### Issues Closes #6732 Closes PY-2574 <!-- junior-request-attribution:start --> via **Ivana Kellyer**. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3AC0BTREZTS2Z%3A1793953060.343649 --> -- [View Session](https://junior-prod.sentry.dev/conversations/slack%3AC0BTREZTS2Z%3A1793953060.343649) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC0BTREZTS2Z%3A1793953060.343649/?project=4510944073809921) <!-- junior-session-footer:end --> --------- Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Ivana Kellyer <ivana.kellyer@sentry.io>
### Description Adds the following attributes: - `faas.invoked_name` (function name), `faas.invoked_provider` (aws), `faas.invoked_region` (region extracted from ARN, else the client's) (all from OTel) on `Invoke`, `InvokeWithResponseStream` and `InvokeAsync` - `aws.lambda.target_function.name` on every other API with `FunctionName` (getsentry/sentry-conventions#716) - `aws.lambda.resource_mapping.id` on the event source mapping APIs, from the request or response `UUID` Other changes: - `FunctionError` sets `error.type=lambda_function_error` - whenever `ClientContext.custom` is available, we add trace propagation headers; trace continuing in aws-lambda integration is not done yet #### Issues Resolves #8069 Part of #8071 --------- Co-authored-by: Ivana Kellyer <ivana.kellyer@sentry.io> Co-authored-by: Alexander Alderman Webb <alexander.webb@sentry.io>
### Description Adds the SQS extension, now SQS MESSAGING operations become `queue.*` spans so they show up in the Queues insight; all other SQS APIs stay `rpc`: - `SendMessage`, `SendMessageBatch`: `queue.publish`, kind `producer` (default is `client`), named `send <queue>` - `ReceiveMessage`: `queue.receive`, named `receive <queue>` - `DeleteMessage`, `DeleteMessageBatch`: `queue.settle`, named `settle <queue>` Adds the following attributes: - `messaging.system` (`aws_sqs`), `messaging.operation.type` and `messaging.operation.name` on the messaging APIs (all from OTel) - `messaging.batch.message_count` on batch APIs: submitted entries for send/delete, returned messages for receive - `messaging.message.id` on `SendMessage`, and on `ReceiveMessage` when exactly one message is returned - `aws.sqs.queue.url` and `messaging.destination.name` (queue name) on every API with `QueueUrl`; destination name also from `QueueName` and `SourceArn` #### Issues Resolves #7572 & PY-2803 --------- Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Johannes Daxböck <114897+dingsdax@users.noreply.github.com> Co-authored-by: Alexander Alderman Webb <alexander.webb@sentry.io> Co-authored-by: Ivana Kellyer <ivana.kellyer@sentry.io> Co-authored-by: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>
### Description Adds the SNS extension to the boto3 integration, now SNS `Publish` and `PublishBatch` become `queue.publish` spans so they show up in the Queues insight; all other operations stay on `rpc`: - `Publish`, `PublishBatch`: `queue.publish`, kind `producer` (default is `client`), named `send <topic>`, or `send <server.address>:<port>` when the request has no destination Adds the following attributes: - `messaging.system` (`aws.sns`), `messaging.operation.type` (`send`) and `messaging.operation.name` (`publish`) on the publish APIs (all from OTel) - `messaging.batch.message_count` on `PublishBatch`, `messaging.message.id` on `Publish` - `aws.sns.topic.arn` and `messaging.destination.name` (topic name) on every API with `TopicArn`, `ResourceArn` or a topic `TargetArn`; destination name also from `CreateTopic`'s `Name` Other changes: - `PhoneNumber` and platform-endpoint `TargetArn` publishes get `messaging.destination.name=[Filtered]` (`send [Filtered]`) #### Issues Resolves #7573 & PY-2804 <!-- junior-request-attribution:start --> via @dingsdax. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3AC0C9X9V5NMT%3A1793904671.980379 --> -- [View Session](https://junior-prod.sentry.dev/conversations/slack%3AC0C9X9V5NMT%3A1793904671.980379) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC0C9X9V5NMT%3A1793904671.980379/?project=4510944073809921) <!-- junior-session-footer:end --> --------- Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Johannes Daxböck <114897+dingsdax@users.noreply.github.com> Co-authored-by: Alexander Alderman Webb <alexander.webb@sentry.io> Co-authored-by: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>
### Description Adds `Boto3Integration(propagate_traces=True)`. Setting it to `False` stops the integration from changing the request params of AWS calls: - Lambda `Invoke` no longer gets trace data in `ClientContext`. - Once #8145 lands: SQS `SendMessage` / `SendMessageBatch` and SNS `Publish` / `PublishBatch` no longer get `sentry-trace` / `baggage` message attributes, and SQS `ReceiveMessage` no longer appends our names to `MessageAttributeNames`. The option gates the shared `get_request_params` hook, so it covers every extension that uses it without per-service changes. Spans are still created as before. This is an escape hatch for setups where the added data breaks calls, e.g. a queue or topic whose `MaximumMessageSize` is set below the 256 KiB the #8145 size check assumes, or consumers that reject unknown attributes. The name matches `CeleryIntegration(propagate_traces=...)`. HTTP `sentry-trace` / `baggage` headers on AWS requests are not affected. They're also added by the stdlib integration and are already controlled by `trace_propagation_targets`. Independent of #8145. The SQS/SNS test cases pass trivially until #8145 is merged, and guard the option after that. Whichever PR merges second needs a small conflict fix at the end of `test_trace_propagation.py`, since both append tests there. <!-- junior-request-attribution:start --> via @sentrivana. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3AC0BTREZTS2Z%3A1794401901.814229 --> -- [View Session](https://junior-prod.sentry.dev/conversations/slack%3AC0BTREZTS2Z%3A1794401901.814229) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC0BTREZTS2Z%3A1794401901.814229/?project=4510944073809921) <!-- junior-session-footer:end --> --------- Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com> Co-authored-by: Ivana Kellyer <ivana.kellyer@sentry.io>
### Description Propagates the trace through SQS and SNS messages, so the consumer can be linked back to the request that sent the message. **Send side** (`SendMessage`, `SendMessageBatch`, `Publish`, `PublishBatch`): - Adds `sentry-trace` and `baggage` as `String` message attributes, on the message or on each batch entry. - Only when `trace_propagation_targets` match the endpoint. - Skips a message if the caller already set these attributes, if it would go over the 10-attribute limit, or if it would go over 256 KiB. AWS would reject the message otherwise. - Works on a copy of the params. The caller's dict isn't changed. **Receive side** (`ReceiveMessage`): - Adds `sentry-trace` and `baggage` to `MessageAttributeNames`, so SQS returns them. Only when the span is recorded, and not when the caller already asks for `All` or `.*`. - Adds a span link to each distinct producer context in the received messages, with `sentry.link.type: message_origin`. - Reads the context from SQS message attributes, and from the JSON envelope of an SNS notification without raw message delivery. **Tests**: injection and extraction for single and batch calls, the limits above, and a round-trip test: send a message, receive what was injected, and check that the link points at the send span. Stacked on #8142 (SNS), which is stacked on #8137 (SQS). #### Issues * resolves: PY-3008 <!-- junior-request-attribution:start --> via @dingsdax. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3AC0C9X9V5NMT%3A1793904671.980379 --> -- [View Session](https://junior-prod.sentry.dev/conversations/slack%3AC0C9X9V5NMT%3A1793904671.980379) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC0C9X9V5NMT%3A1793904671.980379/?project=4510944073809921) <!-- junior-session-footer:end --> --------- Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Johannes Daxböck <114897+dingsdax@users.noreply.github.com> Co-authored-by: Ivana Kellyer <ivana.kellyer@sentry.io> Co-authored-by: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>
Ran a deslop tool on the SDK. Didn't work that great. This is some actionable stuff that it did.
### Description With #8117 (merged), a synchronous boto3 `Invoke` carries `sentry-trace` and `baggage` in `ClientContext.custom`. This PR adds the receiving side: the Lambda integration continues the trace from `context.client_context.custom`. `ClientContext` wins over event headers, because a forwarded HTTP event can still carry headers from further upstream. `sam local` drops `ClientContext`, so the new `ClientContextTrace` test function hands the event's `client_context` to the runtime. That way the container tests can cover it. This is Pablo's code from `pablo/aws-lambda-client-context-trace`. The one change: `client_context` and `custom` are read with `getattr`, so a context object without them falls back to event headers instead of failing. It's the first of three stacked PRs for the stretch Lambda work: this PR → trigger detection (#8222) → `queue.process` spans (#8223). #### Issues * resolves: PY-3009 (receiving side; #8117 does the injection) * Part of #8071 <!-- junior-request-attribution:start --> via @dingsdax. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3AC0C9X9V5NMT%3A1793904671.980379 --> -- [View Session](https://junior-prod.sentry.dev/conversations/slack%3AC0C9X9V5NMT%3A1793904671.980379) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC0C9X9V5NMT%3A1793904671.980379/?project=4510944073809921) <!-- junior-session-footer:end --> --------- Co-authored-by: Pablo Deputter <pablo.deputter@sentry.io> Co-authored-by: Pablo Deputter <71842639+pabloDeputter@users.noreply.github.com> Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Johannes Daxböck <114897+dingsdax@users.noreply.github.com>
) ### Description Since #8137, #8142, and #8145, boto3 puts `sentry-trace` in SQS and SNS message attributes, but nothing reads it on the Lambda side. This PR is the smallest change that closes the loop for SQS- and SNS-triggered functions. The Lambda integration now reads `sentry-trace` from each record and adds a `message_origin` link from the invocation span to each distinct producer span. This matches what the boto3 `ReceiveMessage` span already does. Messages never become the invocation's parent. It reads: - SQS records: `messageAttributes["sentry-trace"].stringValue` - SNS records: `Sns.MessageAttributes["sentry-trace"].Value` - SNS → SQS without raw message delivery: the SNS envelope in the SQS `body`, the same as boto3 does Links are deduplicated, because a batch send gives many messages the same producer span, and capped at 128, OTel's default, because Lambda can deliver up to 10,000 SQS messages at once. **Scope.** This is intentionally kept separate from the stretch stack. It doesn't add `faas.trigger` (#8222) or `queue.process` spans (#8223), and it doesn't touch `messaging.batch.message_count` (#8229). The invocation span is otherwise unchanged. Synchronous `Invoke` propagation through `ClientContext` is #8221. That PR changes the `continue_trace` call and this one only adds `links=`, so the two are independent. The link-building code repeats boto3's `_message_link` in a Lambda-event shape: Lambda records use `messageAttributes` / `stringValue` / `body`, while the boto3 response uses `MessageAttributes` / `StringValue` / `Body`. Importing from the boto3 integration would make `aws_lambda` depend on botocore. **Testing.** `test_producer_links` sends SQS, SNS, and SNS-to-SQS events to `BasicOkSpanStreaming`. It asserts the segment span's `links` and that the segment span has no parent. It passes in CI's Cloud (Lambda) jobs, and `ruff` and `mypy sentry_sdk` pass. `codecov/patch` is red because the Lambda tests run in `sam local` containers, so their coverage isn't counted. #### Issues * Part of #8081 <!-- junior-request-attribution:start --> via **webb**, @sentrivana. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3AC09M26BC6A1%3A1794211222.463239 --> -- [View Session](https://junior-prod.sentry.dev/conversations/slack%3AC09M26BC6A1%3A1794211222.463239) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC09M26BC6A1%3A1794211222.463239/?project=4510944073809921) <!-- junior-session-footer:end --> --------- Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Alexander Webb <alexander.webb@sentry.io> Co-authored-by: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>
… span (#8229) ### Description The Lambda span set `messaging.batch.message_count` to `1` for every event that isn't a list, and to `len(event)` for a top-level list. An SQS or SNS delivery is a dict with `Records`, so a batch of 10 messages reported 1. HTTP calls and direct invocations reported a batch they don't have. This PR removes the attribute from the Lambda span. The count belongs on messaging spans, as in the OTel messaging conventions: - boto3 `ReceiveMessage` and batch-send spans already set it (#8137, #8142). - For SQS/SNS-triggered Lambdas, the `queue.process` span in #8223 sets it. The JS SDK doesn't set it on the Lambda span either. The attribute was added in 3.0 (#7941) and never shipped in 2.x, so no released behavior changes. Replaces #8219 (same commits). GitHub never ran CI on #8219. **Note for review:** this is narrower than PY-3015, which asks to keep the attribute for SQS/SNS deliveries. @alexander-alderman-webb, please confirm that's OK. #### Issues * resolves: PY-3015 <!-- junior-request-attribution:start --> via @dingsdax. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3AC0C9X9V5NMT%3A1793904671.980379 --> -- [View Session](https://junior-prod.sentry.dev/conversations/slack%3AC0C9X9V5NMT%3A1793904671.980379) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC0C9X9V5NMT%3A1793904671.980379/?project=4510944073809921) <!-- junior-session-footer:end --> --------- Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Johannes Daxböck <114897+dingsdax@users.noreply.github.com>
### Description Sets `faas.trigger` on the Lambda span based on the shape of the delivered event: - `http`: API Gateway REST/HTTP, function URLs, ALB, CloudFront, AppSync - `pubsub`: SQS, SNS, Kinesis, Kafka, MQ, SES, EventBridge - `datasource`: S3, DynamoDB Streams, DocumentDB - `timer`: EventBridge schedules - `other`: anything else The trigger is read from the event envelope, never from its payload. `test_trigger` has one case per code path. Other event sources use the same lookup table, so they have no extra cases. This PR doesn't touch `messaging.batch.message_count`. That change is in #8229. This is Pablo's code from `pablo/aws-lambda-trigger`, without his batch counting for the invocation span. It's the second of three stacked PRs (#8221 → this → #8223). #### Issues * resolves: PY-3018 (`faas.trigger` part; the per-source attributes in the issue aren't in this PR) * Part of #8081 <!-- junior-request-attribution:start --> via @dingsdax. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3AC0C9X9V5NMT%3A1793904671.980379 --> -- [View Session](https://junior-prod.sentry.dev/conversations/slack%3AC0C9X9V5NMT%3A1793904671.980379) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC0C9X9V5NMT%3A1793904671.980379/?project=4510944073809921) <!-- junior-session-footer:end --> --------- Co-authored-by: Pablo Deputter <pablo.deputter@sentry.io> Co-authored-by: Pablo Deputter <71842639+pabloDeputter@users.noreply.github.com> Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Johannes Daxböck <114897+dingsdax@users.noreply.github.com>
### Description For an SQS or SNS delivery, the handler now runs inside a `queue.process` child span of kind `consumer`. The span has: - `messaging.system` - `messaging.operation.type` and `messaging.operation.name`, both `process` - the batch message count - the destination name (queue or topic, from the first record) This fills in the processing side of the Queues insight. **Producer links move to this span.** The OTel messaging conventions put a consumer's links to each message's creation context on its process span. #8257 (merged) put them on the invocation span. This PR moves them: the invocation span has no links anymore, and `queue.process` gets them. The links are built by the same `_get_producer_links` from #8257 (dedupe, the cap of 128, skipping the continued trace), with no second copy of that parsing. The tests reuse the `SQS_RECORD` / `SNS_RECORD` fixtures and the `test_producer_links` cases already on `major/3.0`. Those tests now assert the links on the process span and that the invocation span has none. For comparison, the JS SDK's Lambda integration adds no links. It only continues the trace from `ClientContext` or headers. Its links are on the SQS `ReceiveMessage` span only. This is Pablo's code from `pablo/aws-lambda-messaging`, minus his link code. On his branch it sat on top of HTTP and invocation attribute work (PY-3017, PY-3019, PY-3021), which isn't part of the November plan. In `_handler.py` I only wrapped the handler call in the process span. `test_process_span_error` covers a handler that raises. Open points for review: - `messaging.message.id` on single-message process spans isn't added yet (proposed in PY-3020). - The destination name comes from the first record. The alternative is to set it only when all records share it. #### Issues * resolves: PY-3020 * Part of #8083 <!-- junior-request-attribution:start --> via @dingsdax, @sentrivana. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3AC09M26BC6A1%3A1794211222.463239 --> -- [View Session](https://junior-prod.sentry.dev/conversations/slack%3AC09M26BC6A1%3A1794211222.463239) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC09M26BC6A1%3A1794211222.463239/?project=4510944073809921) <!-- junior-session-footer:end --> --------- Co-authored-by: Pablo Deputter <pablo.deputter@sentry.io> Co-authored-by: Pablo Deputter <71842639+pabloDeputter@users.noreply.github.com> Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Johannes Daxböck <114897+dingsdax@users.noreply.github.com> Co-authored-by: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>
We're preparing our next major on this branch.
The project is tracked in Linear. If you don't have access, we'll try to tag issues belonging to the project with the
SDK3.0 label on GitHub so that you can follow along.Notable changes
Context
You might have read this announcement about us discontinuing work on a 3.0. This is referring to the work done on the
potel-basebranch, which included two types of changes: a huge refactor of our tracing code on the one hand, and various unrelated changes, improvements and fixes on the other. We're dropping the huge refactor part, and only porting the rest, to a new branch and eventually a new 3.0 release.