Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 37 additions & 15 deletions HACKING.md
Original file line number Diff line number Diff line change
Expand Up @@ -912,8 +912,25 @@ growing the fixed command or state-slot ABI. The shader bypasses shade,
patterns, alpha comparison, logic, blending and dithering, matching the CPU
mode fold even when stale raw mode bits request those operations.

This completes the M4 ordinary/zoomed pixmap boundary. READ remains a hard
error until M5 supplies an explicit synchronized source.
This completes the M4 ordinary/zoomed pixmap boundary.

Resident HostR uses the existing explicit raw-plane observation boundary. One
reusable CPU snapshot holds only the selected RGB or auxiliary plane and is
keyed by plane, resident epoch and last-write position. A cache miss submits
the exact accepted prefix, maps the fixed readback staging buffer once and
copies the complete selected plane; unchanged generations reuse that snapshot
for every PIO or DMA word and may reuse it across transfer boundaries. The
existing CPU HostR walker reads the snapshot rather than either board CPU
framebuffer, then READ advances the resident stream without changing a plane
generation. Selected-plane writes, reset and restore invalidate the key.

The board interface preserves its asymmetric phase rules: a PIO GO read
returns the current latch and queues the next word, while scalar and bulk DMA
prime before returning data. A bulk token runs and retires its complete HostRW
batch once. Observation counters attribute full-plane attempts, successful
bytes and stream cuts to `HostRead`; snapshot and monitor reasons reuse the
same synchronization primitive. Failures remain sticky and never fall back to
the stale CPU planes. This completes M5 in the gated backend.

Each activation or quiescent reset establishes a new epoch. Typed stream
positions distinguish accepted, submitted and callback-confirmed work; each
Expand All @@ -938,28 +955,33 @@ current VC2 tables leave untouched. An inactive display leaves this history
unchanged. A first resident frame checks the preserved parts of the CPU DID
map once and rejects nonzero history that cannot be reconstructed from
current VC2 state; it does not upload the decoded map. Moving that history
between devices or back to a CPU renderer is not implemented. A mode-table
fence published after the
cut defers the frame rather than forcing fence progress or presenting stale
state. GL and presenters using another device are rejected. Screenshot,
frame request, VNC and headless test capture read the composed output only,
and publish pixels and complete requests only after a successful copy/map.
An ignored hardware test also sends ordered draws and a resize through the
between devices or back to a CPU renderer is not implemented. A mode-table fence published after the cut defers the frame rather than forcing
fence progress or presenting stale state. GL and presenters using another
device are rejected by the direct path. Screenshot, frame request, VNC and
headless test capture read the composed output only, and publish pixels and
complete requests only after successful staging. An ignored hardware test
also sends three ordered presentations, including a resize, through the
common wgpu presenter on a native window using the same device. It compares
the presenter's unpacked frame texture with the software compositor using an
explicit test-only readback. This exercises surface submission but does not
inspect the final swapchain or on-screen pixels. The threaded window test is
available on Windows and Linux; macOS requires a main-thread window harness.
An explicitly selected compatibility display instead attributes a full
two-plane observation to `Compatibility`, copies the 16 MiB snapshot into a
separate CPU screen cache and calls the ordinary renderer. It does not
acknowledge the board's CPU shadows and is never an automatic fallback.
The test-only display selection is a fixed typed `DirectWgpu` or
`ReadbackCompatibility` mode; renderer rejection, capture demand and capture
failure never switch it. The explicitly selected compatibility display
attributes a full two-plane observation to `Compatibility`, copies the 16 MiB
snapshot into a separate CPU screen cache and calls the ordinary renderer. It
does not acknowledge the board's CPU shadows and is never an automatic
fallback. Per-mode composed-capture attempt, success and byte totals remain
separate from attributed raw-plane observation counters and survive display
shutdown and resident reset.
After the initial seed, ordinary resident frames and idle/fence processing
perform no raw-plane transfer, CPU-plane access or CPU raster fallback.
Raw-plane observations explicitly select RGB, auxiliary or both, and return
attributed snapshots; installing and acknowledging a CPU shadow is a separate
operation. Board-level HOSTR, snapshots, recording and other raw consumers are
not wired up while resident. The idle processor polls the device error sink,
operation. Board-level HostR is wired through attributed `HostRead` snapshots;
standalone snapshots, recording and other raw consumers are not wired up while
resident. The idle processor polls the device error sink,
and the first fault is sent once through the machine's fatal channel.
Fatal GPU errors reject reset and restart before touching CPU planes; a
successful quiescent reset after a recoverable rejection rearms notification.
Expand Down
38 changes: 29 additions & 9 deletions TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -290,8 +290,8 @@ asynchronous device-error sink before and after submission and while idle,
including a full GFIFO and a full ordinary machine event queue. They require
a single first-fault notification and reject restart.
These tests opt in to the test-only display worker; they do not enable resident
raster in the emulator. They do not complete board-level HOSTR, snapshot or
other raw consumers, full opcode/lifecycle coverage, or production guest
raster in the emulator. They do not complete board-level snapshot or monitor
raw consumers, full opcode/lifecycle coverage, or production guest
validation. On Windows and Linux, an additional hardware test sends two
ordered draws and a resize through the common wgpu presenter on a native
window. It explicitly reads the presenter's unpacked frame texture, including
Expand Down Expand Up @@ -395,12 +395,22 @@ arena, three state slots and two reusable GPU sets to force repeated payload
and state exhaustion. Complete planes, context, counters and stream positions
must remain CPU-identical on both backends.

Two focused CPU tests pin resident HostR board gating, sticky failure and
batch retirement across PIO, scalar DMA and bulk DMA ports. The hardware
observation case runs the existing CPU HostR walker against one reusable
selected-plane snapshot and checks RGB/auxiliary cache reuse. The board oracle
writes distinct GPU-resident RGB words, verifies PIO read-then-advance and
bulk-DMA prime-then-read values, proves repeated GOs use one attributed
full-plane map, then writes the selected plane and requires exactly one new
map. The board CPU framebuffer remains stale throughout.

```sh
cargo test --locked -p iris --lib --features wgpu,rex-jit,rexdiag resident_ -- --test-threads=1
cargo test --locked -p iris --lib --features wgpu,rex-jit,rexdiag dev::ng1::rex3_flat_fill::tests -- --test-threads=1
cargo test --locked -p iris --lib --features wgpu,rex-jit,rexdiag dev::ng1::wgpu::line::tests -- --test-threads=1
cargo test --locked -p iris --lib --features wgpu,rex-jit,rexdiag dev::ng1::wgpu::host::tests -- --test-threads=1
cargo test --locked -p iris --lib --features wgpu,rex-jit,rexdiag mixed_host_sources_match_between_interpreter_and_jit -- --test-threads=1
cargo test --locked -p iris --lib --features wgpu,rex-jit,rexdiag test_hostr_board_ -- --test-threads=1
cargo test --locked -p iris --lib --features wgpu,rex-jit,rexdiag line_pattern_closed_form_matches_scalar_advancement -- --test-threads=1
cargo test --locked -p iris --lib --features wgpu,rex-jit,rexdiag dev::ng1::wgpu::command::tests -- --test-threads=1
cargo test --locked -p iris --lib --features wgpu,rex-jit,rexdiag direct_ng1_scanout_ -- --test-threads=1
Expand All @@ -425,13 +435,13 @@ Remove-Item Env:\IRIS_NO_JIT -ErrorAction SilentlyContinue
.\scripts\test-resident-gate.ps1
```

The exact 99-case script requires 28 `resident_` CPU tests, four flat-fill tests, two line
tests, two HostW walker tests, one HostW interpreter/JIT source test, one
shade-DDA test, one pattern-state test, three command-ABI tests and three
direct-state tests, then **19 resident GPU tests and eight direct scanout
tests on each backend**: 99 tests in total. The window test requires an
available native desktop and a GPU that can present to it. The script checks
each libtest pass count, rejects
The exact 103-case script requires 28 `resident_` CPU tests, four flat-fill
tests, two line tests, two HostW walker tests, one HostW interpreter/JIT source
test, one shade-DDA test, one pattern-state test, two resident HostR
port/retirement tests, three command-ABI tests and three direct-state tests,
then **19 resident GPU tests and nine direct scanout tests on each backend**:
103 tests in total. The window test requires an available native desktop and a
GPU that can present to it. The script checks each libtest pass count, rejects
hardware skips and prints the selected adapter; both Vulkan and D3D12 must
succeed for the full gate. `.\scripts\test-resident-gate.ps1 -Backend vulkan`
or `-Backend dx12` runs just one hardware backend and is not the complete
Expand All @@ -440,6 +450,16 @@ two-backend gate. For manual PowerShell runs set
instead of the shell prefix. A host that lacks either required hardware
backend cannot claim the complete gate. Metal is not covered by this script.

The direct group covers the explicit resident frontend matrix: CPU
software/GL composition, CPU planes through wgpu scanout, same-device
resident wgpu scanout, and explicitly labeled resident readback through the
ordinary CPU renderer. Both resident modes exercise transactional screenshot,
headless frame and loopback VNC capture. Capture failures must keep tickets
pending, preserve the last published pixels and `frame_seq`, and leave the
selected display mode unchanged. Assertions distinguish per-mode composed
capture attempts, successes and bytes from attributed raw-plane
`Compatibility` observations.

To deliberately regenerate, set `REX3_CORPUS_OUT` to an existing relative
directory with a pre-existing regular `.rex3-corpus-root` file and run
`rex3_corpus_generate -- --ignored --nocapture` with the strict CPU feature
Expand Down
87 changes: 61 additions & 26 deletions clanker/plans/GPUTodo.md
Original file line number Diff line number Diff line change
Expand Up @@ -772,14 +772,14 @@ It is deliberately slow and never an automatic fallback. M2 is complete
within this gated test-only scope. Production live integration and final
on-screen validation remain open; `iris-gui` cannot select this mode.

The Windows gate `.\scripts\test-resident-gate.ps1` checks 28 resident CPU,
4 flat-fill, 2 line, 2 HostW walker, 1 HostW interpreter/JIT source oracle,
1 shade-DDA, 1 pattern-state, 3 command-ABI and 3 direct-state tests, plus 19
resident GPU and 8
The Windows gate `.\scripts\test-resident-gate.ps1` checks 103 cases: 28
resident CPU, 4 flat-fill, 2 line, 1 shade-DDA, 1 pattern-state, 3 command-ABI
and 3 direct-state tests, 2 HostW walker tests, 1 HostW interpreter/JIT source
oracle, 2 resident HostR port/retirement tests, plus 19 resident GPU and 9
direct-scanout tests on each of Vulkan and D3D12. It requires the named
backend on an integrated/discrete adapter and a native desktop, and fails
rather than skipping when unavailable; a single-backend run is not the
complete 99-test gate. See
complete 103-test gate. See
`TESTING.md` for commands and prerequisites.

### M3 - Command ABI and complete uber-shader
Expand Down Expand Up @@ -988,36 +988,71 @@ no per-qword GPU submission. HOSTR remains M5 work.

### M5 - HOSTR and CPU observation

- [ ] Add a fixed reusable full-plane readback path.
- [ ] Synchronize only the selected source plane(s).
- [ ] Run the existing CPU HOSTR walker against the synchronized snapshot.
- [ ] Reuse one snapshot for the full PIO/DMA transfer generation.
- [ ] Preserve PIO read-then-advance and DMA prime-then-read behavior.
- [ ] Reuse the synchronization primitive for snapshots and monitor raw-plane
- [x] Add a fixed reusable full-plane readback path.
- [x] Synchronize only the selected source plane(s).
- [x] Run the existing CPU HOSTR walker against the synchronized snapshot.
- [x] Reuse one snapshot for the full PIO/DMA transfer generation.
- [x] Preserve PIO read-then-advance and DMA prime-then-read behavior.
- [x] Reuse the synchronization primitive for snapshots and monitor raw-plane
reads.
- [ ] Measure before considering region readback.

Complete when HOSTR matches CPU mode and performs at most one plane readback
per transfer generation, never one per qword.

The test-only resident backend already has reusable, explicitly selected raw
plane observation and shadow acknowledgement; none of these checkboxes imply
that the board's HOSTR, snapshot or monitor consumers are wired to it.
- [x] Measure before considering region readback.

The resident raster keeps one bounded full-plane CPU snapshot for the selected
HostR source. Its cache key is the plane plus the resident epoch and last-write
position. A miss uses the existing typed observation path to submit the exact
accepted prefix, copy only RGB or auxiliary through the fixed staging buffer,
and attribute the complete-plane bytes to `HostRead`. The existing CPU HostR
walker then consumes that snapshot; it never receives `fb_rgb` or `fb_aux`.
PIO and bulk DMA GOs sharing an unchanged source generation reuse the same
snapshot, including across transfer boundaries. A selected-plane write,
restore or reset changes the key and forces exactly one new observation.

The board ports retain their architectural phase: PIO reads return the current
latch before a GO advances it, scalar DMA primes before reading, and a bulk DMA
token fills and retires its complete HostRW run once. READ advances the
resident stream without changing a plane generation. CPU port tests pin these
phases and sticky failure, while the Vulkan/D3D12 hardware oracle reads known
GPU-written words through PIO and DMA, proves one map for repeated reads,
forces a second map after a selected-plane write, and verifies that the board
CPU plane remains stale.

M5 is complete within the gated resident backend: HostR matches CPU mode and
performs at most one selected-plane readback per transfer generation, never
one per qword. Snapshot and monitor board consumers remain later explicit
observer/lifecycle work; they already share the same attributed observation
primitive.

### M6 - Explicit observers and frontend matrix

- [ ] Read back composed output for screenshots and VNC/CI capture.
- [ ] Keep raw-plane reads separate from composed-frame reads.
- [ ] Validate CPU+GL, CPU+wgpu, wgpu+wgpu and readback wgpu+GL/CPU modes.
- [ ] Label the readback compositor mode as compatibility/validation only.
- [ ] Ensure capture users cannot silently force the proper GPU path into
- [x] Read back composed output for screenshots and VNC/CI capture.
- [x] Keep raw-plane reads separate from composed-frame reads.
- [x] Validate CPU+GL, CPU+wgpu, wgpu+wgpu and readback wgpu+GL/CPU modes.
- [x] Label the readback compositor mode as compatibility/validation only.
- [x] Ensure capture users cannot silently force the proper GPU path into
continuous raw-plane readback without reporting the mode change.

Complete when all four combinations produce matching images and only the
documented combinations/readers increment readback counters.

Composed-output screenshot, headless and VNC capture are exercised by the
test-only resident display; live frontend integration remains open.
Complete within the gated resident backend. The test-only display has a fixed,
typed `DirectWgpu` or `ReadbackCompatibility` mode. Renderer rejection,
capture demand and capture failure cannot change that selection. Direct
presentation and its screenshot, frame-request, headless and VNC consumers
read back only the composed output; they produce no raw-plane observation.
Compatibility presentation performs an attributed two-plane
`Compatibility` observation, then uses a separate CPU cache and the ordinary
renderer. Per-mode composed-capture attempts, successes and bytes are counted
separately from typed raw-plane observation traffic and survive display
shutdown and resident reset.

The dual-backend matrix compares CPU software/GL composition, CPU planes
through wgpu scanout, resident planes through same-device wgpu scanout and
resident readback through the ordinary CPU renderer. Capture tickets are
coalesced at the frame cut, newer tickets remain pending, and failures neither
publish stale pixels nor advance `frame_seq`. The complete screenshot,
headless frame and loopback VNC cases run in both resident modes. Live
frontend integration remains open; no production selector exposes either
resident mode yet.

### M7 - Snapshot, reset and failure lifecycle

Expand Down
3 changes: 2 additions & 1 deletion scripts/test-resident-gate.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -71,14 +71,15 @@ try {
Invoke-ResidentTests 'dev::ng1::wgpu::line::tests' 2 $false
Invoke-ResidentTests 'dev::ng1::wgpu::host::tests' 2 $false
Invoke-ResidentTests 'mixed_host_sources_match_between_interpreter_and_jit' 1 $false
Invoke-ResidentTests 'test_hostr_board_' 2 $false
Invoke-ResidentTests 'dev::ng1::wgpu::shade::tests' 1 $false
Invoke-ResidentTests 'line_pattern_closed_form_matches_scalar_advancement' 1 $false
Invoke-ResidentTests 'dev::ng1::wgpu::command::tests' 3 $false
Invoke-ResidentTests 'direct_ng1_scanout_' 3 $false
foreach ($name in $requestedBackends) {
$env:REX3_RESIDENT_BACKEND = $name
Invoke-ResidentTests 'resident_gpu_' 19 $true
Invoke-ResidentTests 'dev::ng1::wgpu::direct_tests' 8 $true
Invoke-ResidentTests 'dev::ng1::wgpu::direct_tests' 9 $true
}
} finally {
[Environment]::SetEnvironmentVariable('REX3_RESIDENT_BACKEND', $previousBackend, 'Process')
Expand Down
Loading