Skip to content

vpage: fix free tracking, release partial maps, enable vpages on small-L2 (NVL-S) - #11303

Merged
kv2019i merged 4 commits into
thesofproject:mainfrom
lgirdwood:fix/vpage-region-underflow
Nov 10, 2026
Merged

kv2019i merged 4 commits into
thesofproject:mainfrom
lgirdwood:fix/vpage-region-underflow

Conversation

@lgirdwood

@lgirdwood lgirdwood commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

Follow-up to the vpage overflow check that landed in 27eec27 (#11323). The Zephyr TLB driver now maps the uncached alias (zephyrproject-rtos/zephyr#122801, in the current west.yml pin), so vpages beyond physical L2 can be used on NVL-S.

  1. vpage: fix element tracking when freeing a non-last allocation - velems[num_elems_in_use] (one past the last element) was moved over the released slot, orphaning the last element and leaking its pages. Adds an out-of-order free boot test.
  2. vpage: release pages mapped by a failed multi-page map - the mm driver leaves already mapped pages (and their physical pages) mapped on failure.
  3. vpage: size the vpage region without underflow, support small L2 - replaces the "disable vpages" fallback with a layout for small L2: place the region after the virtual heap, end it at the library base (or end of virtual memory), cap it by the physical L2 left after the image, and disable vpages only if nothing is left.

Rebased onto main; the DNM west.yml commit is dropped since the Zephyr fix is already in the pin. vpage.c compiles for nvl-s; not yet run on NVL-S hardware, which is what the CI run should confirm. Draft.

🤖 Generated with Claude Code

@lgirdwood

Copy link
Copy Markdown
Member Author

@harajend fyi - pls let me know if this helps.

@lgirdwood
lgirdwood marked this pull request as ready for review October 13, 2026 17:10
Copilot AI balanced review requested due to automatic review settings October 13, 2026 17:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Region boundary handling still has edge-case bugs, and the new regression test does not detect the original leak.

3 open findings
What changed in this PR

Fixes vpage sizing and allocation tracking, including NVL-S support.

Changes:

  • Prevents virtual-region size underflow.
  • Cleans up partially mapped pages and fixes out-of-order frees.
  • Adds an out-of-order free boot test.
File Description
zephyr/​lib/​vpage.c Updates region sizing, cleanup, and allocation tracking.
zephyr/​test/​vpage.c Adds an out-of-order free regression test.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread zephyr/lib/vpage.c Resolved
Comment thread zephyr/lib/vpage.c Resolved
Comment thread zephyr/test/vpage.c Resolved
@lgirdwood
lgirdwood force-pushed the fix/vpage-region-underflow branch from 6115723 to b6f127a Compare October 13, 2026 18:17
@intel-sofci

intel-sofci commented Oct 13, 2026 •

Copy link
Copy Markdown

PR 11303: test results

Run date: 2026-11-10 08:38 UTC

Tested commit: 621fb80f143e865509373482d35cc38bde00de3a

mtl pass rate lnl pass rate ptl pass rate wcl pass rate nvl pass rate

@harajend

harajend commented Oct 15, 2026 •

Copy link
Copy Markdown

@harajend fyi - pls let me know if this helps.

@lgirdwood

Tested commit b6f127a46 via intel-innersource CI run
38693960124
on both NVL-S DUTs (ba-nvls-rvp-01, ba-nvls-rvp-02).

This issue is not fixed. All playback/capture tests still fail on both
boards with a DSP firmware crash (EXCCAUSE 29 store prohibited) triggered
on the first pipeline-create IPC, halting DSP core 0 with no recovery.

Detailed analysis:

CI failure: DSP firmware crash on NVL-S (regression vs. previously-reported bug)

Tested commit b6f127a46 via intel-innersource CI run
38693960124
on both NVL-S DUTs (ba-nvls-rvp-01 SoundWire, ba-nvls-rvp-02 HDA).

Result: All playback/capture-dependent test cases fail on both boards with an
identical, reproducible DSP crash triggered by the very first pipeline-create IPC:

<inf> [core 0] vregion.vregion_create: new at base 0xa023a000 size 0x12000 pages 18 metadata at 0x400fdd80
<inf> [core 0] ipc.ipc_cmd: rx: 0x40000004|0x15
<err> [core 0] os.print_fatal_exception:  ** FATAL EXCEPTION
<err> [core 0] os.print_fatal_exception:  ** CPU 0 EXCCAUSE 29 (store prohibited)
<err> [core 0] os.print_fatal_exception:  **  PC 0xa007ef38 VADDR 0xa0031050
...
<err> [core 0] os.z_fatal_error: >>> ZEPHYR FATAL ERROR 0: CPU exception on CPU 0
<err> [core 0] zephyr.k_sys_fatal_error_handler: Halting system

Identical PC/backtrace/address on both boards → deterministic firmware bug, not
hardware/board-specific noise. DSP core 0 halts and never recovers (unlike a
suspend/resume cycle, which does not bring it back here).

This looks like a regression against a previously-filed bug, not a fix

We have an existing internal bug report (BUG_REPORT_NVL-S_vpage_alloc.md,
CI run 38194969857, firmware beeac4752) describing a failure at the
exact same virtual region 0xa023a000:

<err> [core 0] vpage.vpages_alloc_and_map: error: failed to map virtual region 0xa023a000 ..., error -22
<err> [core 0] vpage.vpage_alloc: vpage_alloc failed -22 for N pages, total 1048294 free 1048294
<err> [core 0] pipe.pipeline_new: Failed to create pipeline vregion of X bytes, using heap

In that prior report, the failure was graceful: vpage_alloc() returned
-22 (EINVAL), the firmware fell back to heap allocation, and the DSP stayed
alive (recoverable via a suspend/resume D0→D3→D0 cycle).

With this PR, the same 0xa023a000 region now produces a hard CPU
exception (store prohibited) and a full core halt
, with no recovery path
short of a host reboot.

Hypothesis: this PR enables actually mapping/using the vpage region
instead of returning EINVAL, but something writes into 0xa023a000 /
VADDR 0xa0031050 without the mapping having succeeded (or before it's
committed), so the previously-tolerated error case is now a hard store fault
instead of a handled error return.

Suggested next step

Please check the map-then-use ordering / error handling around
vregion_create() / vpages_alloc_and_map() for the small-L2 (NVL-S) path —
specifically whether the metadata write (metadata at 0x400fdd80) or first
use of the new vregion happens before/without confirming the map succeeded.

@kv2019i kv2019i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a good and important fix, but the commits are mixing multiple issues and as @harajend is reporting functional issues as well, marking it with -1 for now.

Comment thread zephyr/lib/vpage.c Resolved
Comment thread zephyr/lib/vpage.c Resolved
Comment thread zephyr/lib/vpage.c Resolved
@kv2019i

kv2019i commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator

This PR fails without Zephyr zephyrproject-rtos/zephyr#122801 .

@lgirdwood I pushed a minimal/subset of this PR as #11323 -> just fixes the overflow and limits SOF to what is safe to do with current version of Zephyr drivers.

@lgirdwood

Copy link
Copy Markdown
Member Author

This PR fails without Zephyr zephyrproject-rtos/zephyr#122801 .

@lgirdwood I pushed a minimal/subset of this PR as #11323 -> just fixes the overflow and limits SOF to what is safe to do with current version of Zephyr drivers.

Will get Claude to respin once Zephyr part merged.

@lgirdwood lgirdwood changed the title vpage: fix region size underflow, free tracking, and enable vpages on NVL-S vpage: fix free tracking and region sizing, enable vpages on NVL-S Oct 21, 2026
@lgirdwood
lgirdwood force-pushed the fix/vpage-region-underflow branch from b6f127a to 81400fb Compare October 21, 2026 16:40
@lgirdwood

Copy link
Copy Markdown
Member Author

Respun onto current main (commits split per review) and added a DNM west.yml commit pinning zephyr to kv2019i's #122801 head (7280c15b8cff, one commit on top of the current pin) so CI picks up the uncached-alias fix via west. Will drop it and bump the pin when the Zephyr PR merges. Related: #11323 is the minimal subset.

@kv2019i

kv2019i commented Oct 30, 2026

Copy link
Copy Markdown
Collaborator

@lgirdwood Needs a rebase, dependencies should be merged now

@lgirdwood
lgirdwood force-pushed the fix/vpage-region-underflow branch from 81400fb to 176682b Compare October 30, 2026 13:41
@lgirdwood lgirdwood changed the title vpage: fix free tracking and region sizing, enable vpages on NVL-S vpage: fix free tracking, release partial maps, enable vpages on small-L2 (NVL-S) Oct 30, 2026
@lgirdwood

Copy link
Copy Markdown
Member Author

Rebased onto main (the overflow check from #11323 is now upstream, so commit 3 is reduced to enabling the small-L2 layout) and dropped the DNM west.yml commit, as the Zephyr fix is already in the pinned revision (76280bce2a05 contains 5d0c907ec5d). Needs a CI run on NVL-S to confirm.

@kv2019i

kv2019i commented Oct 30, 2026

Copy link
Copy Markdown
Collaborator

@lgirdwood There's no NVL-S in PR CI, so either you need to test this manually before merge, and/or merge and check next daily (there's NVL-S in daily).

@lgirdwood
lgirdwood force-pushed the fix/vpage-region-underflow branch from 176682b to b925b3b Compare October 30, 2026 20:48

@kv2019i kv2019i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some more comments inline

Comment thread zephyr/lib/vpage.c Resolved
Comment thread zephyr/lib/vpage.c Outdated
* region (or the end of virtual memory) and cap it below by the
* physical L2 left after the image, as vpages can never use more.
* Pages beyond physical L2 are accessed through the coherent alias
* too, which needs the Zephyr TLB driver to map the uncached alias.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last sentence is not super useful. More useful would be to replace with a link to the Zephyr PR -- the main point is Zephyr had a bug in using addresses beyond physical L2 end.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, the comment now points at the Zephyr PR (zephyrproject-rtos/zephyr#122801) and states the main point that using pages beyond physical L2 needs the uncached alias mapped.

Comment thread zephyr/lib/vpage.c
@@ -312,18 +296,39 @@ __cold static int vpage_init(void)
end = l2_end - CONFIG_LIBRARY_REGION_SIZE;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a blocker but I wonder if the vpages should go beyond l2_end in this case. IOW why limit vpages to physical L2 size in this case, if we already know Zephyr supports going beyond and we use it below for systems with smaller L2. IOW, shouldn't we size vpages based on a) physical L2 size and b) image size. vpages cannot be larger than a-b, but I don't see a need to limit otherwise.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only cap applied in the small-L2 branch is a) physical L2 minus b) the image size (L2 end - image end), i.e. exactly the bound you describe; beyond that the region extends up to the library base / end of virtual memory. I kept the first branch (everything fits in L2) as is to not change the layout of the other platforms. Happy to unify the two if you prefer, but that would move other platforms' vpage regions.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My comment was specifically about the first branch. Given the code now "oversubscribes" vpage+libregion in case of a smaller L2, why not oversubscribe also in the generic case?

But agreed this is probably handled in a separate PR.

lrgirdwo and others added 3 commits November 9, 2026 14:02
vpages_free_and_unmap() moved velems[num_elems_in_use], which is one past
the last in-use element, over the released slot. Freeing a non-last
allocation therefore left the last element outside the in-use count, so
the next allocation overwrote its tracking entry and that allocation could
never be freed, leaking its virtual and physical pages.

Decrement the count first so the real last element is moved. Add a boot
test that frees allocations out of order and checks that the pages can be
reused.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
The mm driver maps page by page and leaves the pages already mapped, and
their physical pages, in place when a later page fails, e.g. when physical
L2 is exhausted. vpage then freed the virtual blocks without unmapping
them, leaking physical L2.

Unmap any page of the failed range that is still mapped before freeing the
virtual blocks.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
The vpage region size was computed as an unsigned subtraction. The
overflow is now caught, but vpages are then simply disabled when the
virtual heap and library regions do not fit in L2 together with the
image, as on NVL-S (2MB L2).

Those regions are only virtual address space backed on demand, and the
Zephyr TLB driver now maps the uncached alias of pages beyond physical L2
too. When they do not fit, place the vpage region after the virtual heap,
end it at the library base address (or the end of virtual memory), and
cap it by the physical L2 left after the image, as vpages cannot use more
than that. Disable vpages only if no page is left.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
@lgirdwood
lgirdwood force-pushed the fix/vpage-region-underflow branch from b925b3b to c0936fc Compare November 9, 2026 14:03
@lgirdwood

Copy link
Copy Markdown
Member Author

Pushed a fix for the review comments (rollback hunk restored, comment updated). Agreed that NVL-S needs a manual test before merge or a check in the next daily; I have not run it on NVL-S hardware, only compiled it.

@lgirdwood

Copy link
Copy Markdown
Member Author

@kv2019i NVL DUT 03 look stuck - @harajend can you check. Thanks !

@lgirdwood lgirdwood left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pls check we have a ztest for each vpage API and if any missing lets add them. We need to check for over commit, leakage and sparse/continuous mappings.

Add boot tests for vpage_alloc() and vpage_free():
- zero page allocation
- alignment, contiguity and full read/write of a multi-page allocation
- over commit: more than available, absurd sizes and exhausting all pages
  fail cleanly and consume nothing
- leaks: repeated alloc/free of mixed sizes and orders returns every page,
  detected by comparing the largest allocatable block before and after
- sparse maps: holes left by frees are reused by single pages, larger
  blocks never overlap live pages and data is never corrupted
- invalid frees (NULL, unaligned, mid-allocation, double free) are
  rejected without disturbing valid allocations
- exhausting the allocation tracking elements fails cleanly and recovers

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
@lgirdwood

Copy link
Copy Markdown
Member Author

Added a ztest commit. vpage has two APIs, vpage_alloc() and vpage_free(); the boot tests now cover: zero-page alloc, alignment/contiguity/read-write of multi-page blocks, over commit (more than available, absurd sizes, exhausting everything, nothing consumed on failure), leaks (mixed sizes/orders over many rounds, compared via the largest allocatable block before/after), sparse maps (holes reused, larger blocks never overlap live pages, no data corruption), invalid frees (NULL, unaligned, mid-allocation, double free) and exhausting the tracking elements. Compile-checked only; they have not been run on a DUT yet.

Comment thread zephyr/lib/vpage.c
@@ -312,18 +296,39 @@ __cold static int vpage_init(void)
end = l2_end - CONFIG_LIBRARY_REGION_SIZE;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My comment was specifically about the first branch. Given the code now "oversubscribes" vpage+libregion in case of a smaller L2, why not oversubscribe also in the generic case?

But agreed this is probably handled in a separate PR.

@kv2019i
kv2019i merged commit 4a5c1f3 into thesofproject:main Nov 10, 2026
44 of 45 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants