Skip to content

fix(profiler): recover arm64 native stacks across sampling signals - #917

Draft
kaahos wants to merge 13 commits into
paul.fournillon/fix-continuation-carrier-recoveryfrom
paul.fournillon/test-continuation-remount-chaos
Draft

kaahos wants to merge 13 commits into
paul.fournillon/fix-continuation-carrier-recoveryfrom
paul.fournillon/test-continuation-remount-chaos

Conversation

@kaahos

@kaahos kaahos commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?:
Preserve native and Java ancestry during ARM64 CPU and wall profiling. Correct DWARF return-address rules, account for temporary C1 argument/save areas and JNI wrapper frame phases, and recover saved register contexts at Linux vDSO signal returns.

Add deterministic native regressions, a focused GraalVM native-transition regression, and remount chaos coverage shared with the standalone stress harness.

Motivation:
Native leaves can keep their return address in LR rather than a stack slot. Applying the wrong initial or restored DWARF rule loses callers or invents a caller from stale stack contents. C1 runtime calls can also leave temporary argument/save areas outside the compiled frame size. Object-arraycopy subtype calls keep two such areas live.

JNI wrappers restore SP before checking pending exceptions. Applying the full frame size in that window skips the real caller. A bounded instruction decoder recovers the remaining saved pair or live LR according to the interrupted phase. The prologue also accounts for the saved pair before local stack allocation.

Linux ARM64 omits CFI for its vDSO signal-return trampoline. Nested CPU and wall signals need its saved exact PC and register context to resume a correct walk. Linux also supports custom signal restorers; unsupported restorers without signal CFI must stop with truthful truncation.

Additional Notes:

  • Stacked on feat(profiler): recover carrier ancestry at continuation boundaries #887. Heap-resident frozen Java caller recovery remains in fix(profiler): recover frozen virtual-thread callers #879.
  • This PR owns the native DWARF, signal-restorer, generic C1 stack adjustments and JNI wrapper phase recovery. fix(profiler): recover frozen virtual-thread callers #879 inherits these corrections and their tests through its base.
  • Custom-restorer instruction probes require HotSpot/OpenJ9 safefetch recovery; other VMs retain their normal CFI/FP fallback.
  • Context recovery requires the known vDSO, the exact syscall instruction pair, bounded signal-frame reads, forward aligned SP and instruction-aligned PC. An exact custom rt_sigreturn sequence without signal CFI stops with truncation. Signal CFI takes precedence; unrelated syscalls retain normal unwinding.
  • Existing 99% success, ancestry and truthful-completeness gates remain unchanged. The focused Graal regression requires exercised native transition frames.
  • Static Google Test archive ordering permits architecture-gated native test files to link on other architectures.
  • No allocation, blocking synchronization or ucontext mutation is added to signal paths.

How to test the change?:

./.claude/commands/build-and-summarize :ddprof-lib:gtestDebug
JAVA_TEST_HOME=/path/to/graalvm \
  ./.claude/commands/build-and-summarize :ddprof-test:testDebug \
  '-Ptests=*NativeUnwindRegressionTest' -Pskip-gtest -PkeepJFRs
JAVA_TEST_HOME=/path/to/graalvm \
  ./.claude/commands/build-and-summarize :ddprof-test:testDebug \
  -Ptests=VirtualThreadCarrierChaosTest -Pskip-gtest -PkeepJFRs

Validation for signed bbe09e6d8, based on #887 at 50d2704d2.

Configuration Check State Result
macOS ARM64 Full native Debug and Release suites passed 2,225 passes across both configurations
Linux ARM64 Full native ASan/UBSan and Release suites passed Both complete suites passed
ARM64 native Wrapper phase regressions passed 57 production-walker and 10 helper cases in Debug and ASan; controls cover authentication, bounds and walked return addresses
Linux ARM64, HotSpot 21 Uncached CI-mode virtual-thread tests, Debug and ASan passed Each: 13 passed, six existing skips; includes unchanged HighArity and chaos assertions
Linux ARM64, HotSpot 25 GA Uncached CI-mode virtual-thread tests passed 12 passed, seven existing skips
Linux ARM64, GraalVM 21.0.8 Uncached CI-mode native and virtual-thread tests passed Four native cases; 13 virtual-thread cases passed, six existing skips
Linux ARM64, GraalVM 25 GA Uncached CI-mode native and virtual-thread tests passed Four native cases; 12 virtual-thread cases passed, seven existing skips
Linux x64 Architecture-gated test archive-link control passed Old ordering fails; corrected ordering links and runs the empty ARM-only source; build delta unchanged
Quality and formatting spotlessApply, qualityChecks and diff checks passed Final source candidate
Source review Full 29-file delta against #887 passed No remaining actionable findings; marker binds the signed head and exact diff
Remote CI Newly pushed bbe09e6d8 inconclusive New matrix is running; local passes do not establish remote success

The JNI-wrapper production-walker regression fails on the prior production code at the pending allocation and three epilogue phases. A separate diagnostic captured the incorrect full-frame advance after SP restoration. The phase defect is confirmed; the historical CI recording did not retain native registers, so its exact interrupted PC remains unproven. Strict Java workload and assertion bytes are unchanged by this correction.

The prior ARM64 release allocator failure is tracked separately by #911. Its test/probe changes do not overlap this delta.

Both GraalVM selections include the unchanged remount chaos checks. Before consolidation, four C1 regressions failed. Independent review exposed two additional cases, each reproduced before correction: the double-save arraycopy layout and a custom restorer without a registered library.

For Datadog employees:

  • If this PR touches code that signs or publishes builds or packages, or handles credentials of any kind, I've requested a security review.
  • This PR doesn't touch any of that.
  • JIRA: PROF-16298

@dd-octo-sts

dd-octo-sts Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

CI Test Results

Run: #40698944892 | Commit: ffa0910 | Duration: 20m 9s (longest job)

✅ All 32 test jobs passed

Status Overview

JDK glibc-aarch64/debug glibc-amd64/debug musl-aarch64/debug musl-amd64/debug
8 - ✅ - -
8-ibm - ✅ - -
8-j9 ✅ ✅ - -
8-librca - - ✅ ✅
8-orcl - ✅ - -
11 - ✅ - -
11-j9 ✅ ✅ - -
11-librca - - ✅ ✅
17 ✅ ✅ - -
17-graal ✅ ✅ - -
17-j9 ✅ ✅ - -
17-librca - - ✅ ✅
21 ✅ ✅ - -
21-graal ✅ ✅ - -
21-librca - - ✅ ✅
25 ✅ ✅ - -
25-graal ✅ ✅ - -
25-librca - - ✅ ✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0


Updated: 2026-10-28 14:06:50 UTC

@dd-octo-sts

dd-octo-sts Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

✅ All 40 integration tests passed

📊 Dashboard · 👷 Pipeline · 📦 bbe09e6d

@kaahos kaahos changed the title test(profiler): share virtual-thread remount chaos coverage fix(profiler): recover arm64 native stacks across sampling signals Oct 28, 2026
@datadog-datadog-us1-prod

datadog-datadog-us1-prod Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Pipelines

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: bbe09e6 | Docs | Give us feedback!

@jbachorik
jbachorik added this pull request to stack #922 October 29, 2026 15:03

private static Map<String, Object> fields(Object... pairs) {
Map<String, Object> values = new HashMap<>();
for (int i = 0; i < pairs.length; i += 2) values.put((String) pairs[i], pairs[i + 1]);

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant