Add HTTP span tags to the AWS Lambda invocation span - #12226
Conversation
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dc12bd5a6a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…bda HTTP tags Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: be7c5bacbf
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
Bracketed IPv6 Host values containing a non-default listener port retain that port, after which URL construction appends x-forwarded-port again. This produces malformed http.url tags for affected ALB invocations.
🤖 Datadog Autotest · Commit be7c5ba · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
|
🤖 Claude (via automated review) Non-blocking perf note, not a request for changes on this PR: On most request paths this would be noise, but Lambda invocation handlers run at whatever rate the function is invoked, so small per-call overhead compounds across the customer's traffic in a way a one-off request path doesn't. Worth a quick follow-up to fold the lowercasing into a single pass (e.g. build the lowercase map directly instead of copying), independent of this PR. |
dougqh
left a comment
There was a problem hiding this comment.
Noted some performance concerns
But given that lambdas probably aren't as hot as conventional Java applications, I don't those are worth blocking on
Case-colliding header names consequently collapse in payload order rather than in the intermediate map's hash order, matching the Lambda Extension. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
jandro996
left a comment
There was a problem hiding this comment.
A couple of questions from a first pass over this - nothing that should hold up the PR, just curious what you think.
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
Build pipeline has failing jobs for b472c86: What to do next?
DetailsSince those jobs are not marked as being allowed to fail, the pipeline will most likely fail. |
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
What Does This Do
Computes the HTTP span tag set for every HTTP-shaped Lambda trigger from the event the tracer already parses for the WAF, and applies it at span creation. Follow-up to #12202, which did the refactor this builds on.
Tags written by the new
LambdaAppSecHandler.applyHttpTags, transcribed fromHttpServerDecorator.doOnRequest:http.methodhttp.urlURIUtils.buildURL(scheme, host, port, path)— no query string, see belowhttp.query.stringrawQueryStringon v2 and Function URL, the rebuilt query elsewhere, whenisHttpServerTagQueryString()http.useragentuser-agentheaderhttp.routeresource, v2routeKeyminus the method prefix, WebSocketrouteKeyhttp.hostnamex-forwarded-host, else the URL hosthttp.status_codestatusCode, on the exit pathSupporting changes:
LambdaEventParsernow extractshost(requestContext.domainName, else theHostheader) androute, reads ALB headers frommultiValueHeaderswhenheadersis absent, and lowercases request header keys at extraction — as the response side and the Lambda Extension both already do. That last one also fixes a latent miss inextractHeadersWithCookies, whose"cookie"lookup never matched an API Gateway v1 styleCookie.rawPath+rawQueryString, so it is used as-is rather than rebuilt fromqueryStringParameters, which comma-joins repeated keys (?a=1&a=2would come back asa=1%2C2) and drops percent-encoded path segments. The other triggers expose no such field and keep the rebuilt path.:portis stripped from theHostheader. An ALB on a non-standard listener sendsexample.com:8080, and the port is tracked separately fromx-forwarded-port, soURIUtils.buildURLwould otherwise emit it twice. IPv6 literals are left alone.LambdaURIDataAdapter.host()returns the event host instead ofnull; without itURIUtils.buildURLomits the authority andhttp.urlwould be a bare path. The scheme derived fromx-forwarded-protois now lowercased and whitelisted tohttp/https, and the default port follows the scheme rather than always being 443 —URIUtils.buildURLsuppresses a port only for 80/httpand 443/https, and compares the scheme exactly.processResponseDatasets the status throughspan.setHttpStatusCodeso it serialises as a string, and sets the error flag fromgetHttpServerErrorStatuses(), matchingHttpServerDecorator.doOnResponseStatusand the Extension'serror=1on a 5xx handler response.Tagging is gated on AppSec being active, as
processRequestStartalready was. The tags are written onto the AppSec-derivedTagContextbeforemergeContexts, soCoreTracercopies them onto the span at creation and nothing has to retain the parsed event across the customer's handler. No changes outside thedatadog.trace.lambdapackage.Motivation
AppSec Milestone 1 requires the invocation span to carry HTTP tags. Today the tracer sets none; everything on the final
aws.lambdaspan comes from the Lambda Extension, whose coverage is uneven — nohttp.routefor Function URL, WebSocket or ALB, nohttp.urlfor ALB, no query string anywhere, and ahttp.user_agentkey that differs fromTags.HTTP_USER_AGENT. This also makes the tags correct in deployments with no extension in the path, such as a remote agent or thesystem-testsjava_lambdaweblog.Additional Notes
http.url,http.methodandhttp.route: itsmeta.extendruns atruntimeDone, after the tracer's meta is merged. Accepted for now — a follow-up extension change to stop overwriting tracer-supplied keys makes these win with no further tracer work. Until then the span also carries bothhttp.useragentandhttp.user_agent.span.kind: server(it changes stats aggregation and entry-span semantics, and the extension already sets it), and the client-IP tag group (http.client_ip,network.client.ip,peer.ipv4/ipv6/port), which would needClientIpAddressResolvermoved fromagent-bootstraptointernal-api. Both are tracked as separate follow-ups.http.url:QueryObfuscator.processTagsobfuscateshttp.query.stringand re-appends it, so doing it here too would double the query.Tags.HTTP_ROUTErather thanHttpResourceDecorator.withRoute, which would also rewrite the resource name and break the extension'sdd-tracer-serverless-spanplaceholder filter.Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issue🤖 Generated with Claude Code