Skip to content

fix(profiler): recover carrier frames without preserved frame pointers - #887

Open
kaahos wants to merge 8 commits into
paul.fournillon/fix-frozen-virtual-thread-truncationfrom
paul.fournillon/fix-continuation-carrier-recovery
Open

kaahos wants to merge 8 commits into
paul.fournillon/fix-frozen-virtual-thread-truncationfrom
paul.fournillon/fix-continuation-carrier-recovery

Conversation

@kaahos

@kaahos kaahos commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?:
Recover carrier ancestry at virtual-thread continuation boundaries without requiring preserved frame-pointer links.

For JVMs without exported ContinuationEntry layout metadata, derive the entry frame from SP, the generated enterSpecial frame size, and stack-argument padding. Account for OSR frames having zero stack arguments and for an entry wrapper generated after profiler initialization. Accept a non-pointer saved carrier FP only when the carrier PC resolves to an nmethod; retain stack bounds and PC validation. Track the argument count before compiled prologue/epilogue shortcuts and when recovering interpreter frames directly from Java frame anchors, so those paths can also reach the return barrier with valid metadata.

Debug unwind-failure diagnostics also use a multi-byte JFR event-size prefix. Long stub names captured by the high-arity regression could otherwise corrupt the event size or abort a debug build while stopping the recording.

Motivation:
The existing blocking virtual-thread carrier test fails in PR #877 CI, with captured stacks stopping at break_cont_carrier_sp.

An isolated JDK 21 reproduction also exposes break_cont_entry_fp. Diagnostic capture on macOS aarch64 found a valid continuation entry whose saved carrier FP was 1: compiled carrier code can use FP as a general-purpose register. Following that value as another frame link, or requiring it to look like a pointer before walking compiled carrier code, loses valid ancestry.

OpenJDK's x86 fast-thaw link handling, entry-frame argument/alignment calculation, and generated entry-frame setup provide the basis for this recovery calculation.

Additional Notes:

How to test the change?:
Two integration regressions require a return barrier, inspect samples with callers still frozen, reject continuation-pointer errors, and require carrier ancestry. One uses register arguments; the other forces stack-passed arguments. Both fail on the unchanged profiler on macOS aarch64/JDK 21; the Linux aarch64 baseline also fails a new regression. Fully thawed vmx coverage and eight carrier-recovery native tests cover entry slots, argument alignment, invalid metadata, overflow, and late wrapper generation.

JAVA_HOME=/path/to/build/jdk JAVA_TEST_HOME=/path/to/test/jdk \
  ./.claude/commands/build-and-summarize :ddprof-test:testDebug \
  '-Ptests=*VirtualThread*Test' -Pskip-gtest -PkeepJFRs

JAVA_HOME=/path/to/build/jdk \
  ./.claude/commands/build-and-summarize :ddprof-lib:gtestDebug_continuationCarrierFrame_ut \
  :ddprof-lib:gtestDebug_unwindFailureEncoding_ut

The shared vthread-remount chaos workload keeps four virtual threads alive, alternates register-only and stack-passed arguments, and repeatedly parks under deep callers before burning CPU. VirtualThreadCarrierChaosTest runs it for six seconds per mode in the ordinary integration suite, checking CPU and wall samples in both vm and vmx. It requires return-barrier hits and worker progress, rejects continuation errors and missing/duplicated/misplaced carrier ancestry, and verifies truncation when outer callers are absent. Unfinished walks are counted separately and must be truncated. The old library fails the new vmx check with 2,623 continuation errors; the vm control passes.

The stress test exposed rare argument-tracking omissions in compiled shortcuts and direct interpreter-anchor recovery. Initial JDK 25 and mixed-chaos recordings retained break_cont_entry_fp; the final runs below exercise the corrected paths without those errors.

Extend the test duration, and repeat with -XX:+PreserveFramePointer as a control:

DDPROF_VTHREAD_STRESS_SECONDS=120 JAVA_TOOL_OPTIONS=-XX:-PreserveFramePointer \
  ./.claude/commands/build-and-summarize :ddprof-test:testDebug \
  -Ptests=VirtualThreadCarrierChaosTest -Pskip-gtest -PkeepJFRs

The workload is also selectable by name in the existing chaos harness, including alongside alloc-storm and classloader-churn. A clean harness exit alone is not a stack-correctness assertion.

Completed validation:

  • macOS aarch64, Temurin 21.0.8+9: 12 integration cases pass; 6 existing mode skips. Extended chaos runs cover 30 seconds per mode; the final source also passes the default-duration chaos test.
  • macOS aarch64, Temurin 25.0.2: 11 integration cases pass; 7 existing skips, including 30 seconds per mode in the final chaos run.
  • Eight carrier-recovery native tests pass on macOS aarch64.
  • Linux aarch64, Temurin 21.0.8+9: both chaos modes pass for 10 seconds each, before the final interpreter-anchor tracking addition. The earlier combined native/integration command also succeeded.
  • Linux x64, Temurin 21.0.8+9 under emulation: both chaos modes pass with frame-pointer preservation disabled and enabled; the final source is rechecked with preservation disabled.
  • macOS aarch64/JDK 21: the affected integration suite also passes with -XX:+PreserveFramePointer before the final interpreter-anchor tracking addition. A 20-second mixed run with remount, allocation and class-loader churn completes; independent JDK JFR inspection finds 3,965 CPU and 8,791 wall workload samples with one carrier, and no continuation-pointer errors.
  • spotlessApply and git diff --check pass.

Additional regression coverage:

  • The high-arity workload parks and runs its sampling window directly inside a generated method with a legal 255-slot signature and no bytecode back edge. It fails against the old argument-limit implementation and passes after the fix.
  • The final standalone JDK 21 run passes all native tests and the affected integration suite (13 passed, 6 existing mode skips).
  • Six deterministic unwind-event encoding cases cover short names, multi-byte sizes, the 255-byte retained-name limit, truncation of longer names, and the boundary to a following event. Four fail with the original encoding; all six pass with the fix.
  • With Fix continuation metadata lookup and frame alignment #870 temporarily composed for validation, the affected suite passes on macOS aarch64 / Temurin 25.0.4.1 and JDK 27: each run has 12 passed and 7 existing skips.

For Datadog employees:

  • This PR doesn't touch code that signs or publishes builds/packages or handles credentials.
  • JIRA: PROF-16298

@dd-octo-sts

dd-octo-sts Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

CI Test Results

Run: #40016862638 | Commit: 114c695 | Duration: 21m 18s (longest job)

✅ All 32 test jobs passed

Status Overview

JDK glibc-aarch64/debug glibc-amd64/debug musl-aarch64/debug musl-amd64/debug
8 - ✅ - -
8-ibm - ✅ - -
8-j9 ✅ ✅ - -
8-librca - - ✅ ✅
8-orcl - ✅ - -
11 - ✅ - -
11-j9 ✅ ✅ - -
11-librca - - ✅ ✅
17 ✅ ✅ - -
17-graal ✅ ✅ - -
17-j9 ✅ ✅ - -
17-librca - - ✅ ✅
21 ✅ ✅ - -
21-graal ✅ ✅ - -
21-librca - - ✅ ✅
25 ✅ ✅ - -
25-graal ✅ ✅ - -
25-librca - - ✅ ✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0


Updated: 2026-10-23 20:37:29 UTC

@datadog-official

datadog-official Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Pipelines

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 11d7cc1 | Docs | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

✅ All 40 integration tests passed

📊 Dashboard · 👷 Pipeline · 📦 11d7cc1d

@dd-octo-sts

dd-octo-sts Bot commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Reliability & Chaos Results

✅ All reliability & chaos checks passed Pipeline: https://gitlab.ddbuild.io/DataDog/java-profiler/-/pipelines/146779382

…nto paul.fournillon/fix-continuation-carrier-recovery
@kaahos
kaahos marked this pull request as ready for review October 22, 2026 16:52
@kaahos
kaahos requested a review from a team as a code owner October 22, 2026 16:52
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-22T17:00:27.691864Z d65cc18 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@datadog-official datadog-official Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

The 128-word argument limit rejects legal high-arity compiled methods at continuation return barriers, causing break_cont_entry_fp and dropping carrier ancestry.

Open Bits AI session

🤖 Bits Code Review · Commit d65cc18

Comment thread ddprof-lib/src/main/cpp/hotspot/vmStructs.h Outdated
Preserve SP-based carrier recovery with explicit continuation-boundary classification and truncation. Adapt the native walker fixtures to the required metadata, cover missing-metadata rejection, and retain vm/vmx coverage for never-parked virtual threads.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant