diff --git a/HACKING.md b/HACKING.md index d2caa5f0d..48b7d78fc 100644 --- a/HACKING.md +++ b/HACKING.md @@ -912,8 +912,25 @@ growing the fixed command or state-slot ABI. The shader bypasses shade, patterns, alpha comparison, logic, blending and dithering, matching the CPU mode fold even when stale raw mode bits request those operations. -This completes the M4 ordinary/zoomed pixmap boundary. READ remains a hard -error until M5 supplies an explicit synchronized source. +This completes the M4 ordinary/zoomed pixmap boundary. + +Resident HostR uses the existing explicit raw-plane observation boundary. One +reusable CPU snapshot holds only the selected RGB or auxiliary plane and is +keyed by plane, resident epoch and last-write position. A cache miss submits +the exact accepted prefix, maps the fixed readback staging buffer once and +copies the complete selected plane; unchanged generations reuse that snapshot +for every PIO or DMA word and may reuse it across transfer boundaries. The +existing CPU HostR walker reads the snapshot rather than either board CPU +framebuffer, then READ advances the resident stream without changing a plane +generation. Selected-plane writes, reset and restore invalidate the key. + +The board interface preserves its asymmetric phase rules: a PIO GO read +returns the current latch and queues the next word, while scalar and bulk DMA +prime before returning data. A bulk token runs and retires its complete HostRW +batch once. Observation counters attribute full-plane attempts, successful +bytes and stream cuts to `HostRead`; snapshot and monitor reasons reuse the +same synchronization primitive. Failures remain sticky and never fall back to +the stale CPU planes. This completes M5 in the gated backend. Each activation or quiescent reset establishes a new epoch. Typed stream positions distinguish accepted, submitted and callback-confirmed work; each @@ -938,28 +955,33 @@ current VC2 tables leave untouched. An inactive display leaves this history unchanged. A first resident frame checks the preserved parts of the CPU DID map once and rejects nonzero history that cannot be reconstructed from current VC2 state; it does not upload the decoded map. Moving that history -between devices or back to a CPU renderer is not implemented. A mode-table -fence published after the -cut defers the frame rather than forcing fence progress or presenting stale -state. GL and presenters using another device are rejected. Screenshot, -frame request, VNC and headless test capture read the composed output only, -and publish pixels and complete requests only after a successful copy/map. -An ignored hardware test also sends ordered draws and a resize through the +between devices or back to a CPU renderer is not implemented. A mode-table fence published after the cut defers the frame rather than forcing +fence progress or presenting stale state. GL and presenters using another +device are rejected by the direct path. Screenshot, frame request, VNC and +headless test capture read the composed output only, and publish pixels and +complete requests only after successful staging. An ignored hardware test +also sends three ordered presentations, including a resize, through the common wgpu presenter on a native window using the same device. It compares the presenter's unpacked frame texture with the software compositor using an explicit test-only readback. This exercises surface submission but does not inspect the final swapchain or on-screen pixels. The threaded window test is available on Windows and Linux; macOS requires a main-thread window harness. -An explicitly selected compatibility display instead attributes a full -two-plane observation to `Compatibility`, copies the 16 MiB snapshot into a -separate CPU screen cache and calls the ordinary renderer. It does not -acknowledge the board's CPU shadows and is never an automatic fallback. +The test-only display selection is a fixed typed `DirectWgpu` or +`ReadbackCompatibility` mode; renderer rejection, capture demand and capture +failure never switch it. The explicitly selected compatibility display +attributes a full two-plane observation to `Compatibility`, copies the 16 MiB +snapshot into a separate CPU screen cache and calls the ordinary renderer. It +does not acknowledge the board's CPU shadows and is never an automatic +fallback. Per-mode composed-capture attempt, success and byte totals remain +separate from attributed raw-plane observation counters and survive display +shutdown and resident reset. After the initial seed, ordinary resident frames and idle/fence processing perform no raw-plane transfer, CPU-plane access or CPU raster fallback. Raw-plane observations explicitly select RGB, auxiliary or both, and return attributed snapshots; installing and acknowledging a CPU shadow is a separate -operation. Board-level HOSTR, snapshots, recording and other raw consumers are -not wired up while resident. The idle processor polls the device error sink, +operation. Board-level HostR is wired through attributed `HostRead` snapshots; +standalone snapshots, recording and other raw consumers are not wired up while +resident. The idle processor polls the device error sink, and the first fault is sent once through the machine's fatal channel. Fatal GPU errors reject reset and restart before touching CPU planes; a successful quiescent reset after a recoverable rejection rearms notification. diff --git a/TESTING.md b/TESTING.md index cf6ef4257..56aedb53d 100644 --- a/TESTING.md +++ b/TESTING.md @@ -290,8 +290,8 @@ asynchronous device-error sink before and after submission and while idle, including a full GFIFO and a full ordinary machine event queue. They require a single first-fault notification and reject restart. These tests opt in to the test-only display worker; they do not enable resident -raster in the emulator. They do not complete board-level HOSTR, snapshot or -other raw consumers, full opcode/lifecycle coverage, or production guest +raster in the emulator. They do not complete board-level snapshot or monitor +raw consumers, full opcode/lifecycle coverage, or production guest validation. On Windows and Linux, an additional hardware test sends two ordered draws and a resize through the common wgpu presenter on a native window. It explicitly reads the presenter's unpacked frame texture, including @@ -395,12 +395,22 @@ arena, three state slots and two reusable GPU sets to force repeated payload and state exhaustion. Complete planes, context, counters and stream positions must remain CPU-identical on both backends. +Two focused CPU tests pin resident HostR board gating, sticky failure and +batch retirement across PIO, scalar DMA and bulk DMA ports. The hardware +observation case runs the existing CPU HostR walker against one reusable +selected-plane snapshot and checks RGB/auxiliary cache reuse. The board oracle +writes distinct GPU-resident RGB words, verifies PIO read-then-advance and +bulk-DMA prime-then-read values, proves repeated GOs use one attributed +full-plane map, then writes the selected plane and requires exactly one new +map. The board CPU framebuffer remains stale throughout. + ```sh cargo test --locked -p iris --lib --features wgpu,rex-jit,rexdiag resident_ -- --test-threads=1 cargo test --locked -p iris --lib --features wgpu,rex-jit,rexdiag dev::ng1::rex3_flat_fill::tests -- --test-threads=1 cargo test --locked -p iris --lib --features wgpu,rex-jit,rexdiag dev::ng1::wgpu::line::tests -- --test-threads=1 cargo test --locked -p iris --lib --features wgpu,rex-jit,rexdiag dev::ng1::wgpu::host::tests -- --test-threads=1 cargo test --locked -p iris --lib --features wgpu,rex-jit,rexdiag mixed_host_sources_match_between_interpreter_and_jit -- --test-threads=1 +cargo test --locked -p iris --lib --features wgpu,rex-jit,rexdiag test_hostr_board_ -- --test-threads=1 cargo test --locked -p iris --lib --features wgpu,rex-jit,rexdiag line_pattern_closed_form_matches_scalar_advancement -- --test-threads=1 cargo test --locked -p iris --lib --features wgpu,rex-jit,rexdiag dev::ng1::wgpu::command::tests -- --test-threads=1 cargo test --locked -p iris --lib --features wgpu,rex-jit,rexdiag direct_ng1_scanout_ -- --test-threads=1 @@ -425,13 +435,13 @@ Remove-Item Env:\IRIS_NO_JIT -ErrorAction SilentlyContinue .\scripts\test-resident-gate.ps1 ``` -The exact 99-case script requires 28 `resident_` CPU tests, four flat-fill tests, two line -tests, two HostW walker tests, one HostW interpreter/JIT source test, one -shade-DDA test, one pattern-state test, three command-ABI tests and three -direct-state tests, then **19 resident GPU tests and eight direct scanout -tests on each backend**: 99 tests in total. The window test requires an -available native desktop and a GPU that can present to it. The script checks -each libtest pass count, rejects +The exact 103-case script requires 28 `resident_` CPU tests, four flat-fill +tests, two line tests, two HostW walker tests, one HostW interpreter/JIT source +test, one shade-DDA test, one pattern-state test, two resident HostR +port/retirement tests, three command-ABI tests and three direct-state tests, +then **19 resident GPU tests and nine direct scanout tests on each backend**: +103 tests in total. The window test requires an available native desktop and a +GPU that can present to it. The script checks each libtest pass count, rejects hardware skips and prints the selected adapter; both Vulkan and D3D12 must succeed for the full gate. `.\scripts\test-resident-gate.ps1 -Backend vulkan` or `-Backend dx12` runs just one hardware backend and is not the complete @@ -440,6 +450,16 @@ two-backend gate. For manual PowerShell runs set instead of the shell prefix. A host that lacks either required hardware backend cannot claim the complete gate. Metal is not covered by this script. +The direct group covers the explicit resident frontend matrix: CPU +software/GL composition, CPU planes through wgpu scanout, same-device +resident wgpu scanout, and explicitly labeled resident readback through the +ordinary CPU renderer. Both resident modes exercise transactional screenshot, +headless frame and loopback VNC capture. Capture failures must keep tickets +pending, preserve the last published pixels and `frame_seq`, and leave the +selected display mode unchanged. Assertions distinguish per-mode composed +capture attempts, successes and bytes from attributed raw-plane +`Compatibility` observations. + To deliberately regenerate, set `REX3_CORPUS_OUT` to an existing relative directory with a pre-existing regular `.rex3-corpus-root` file and run `rex3_corpus_generate -- --ignored --nocapture` with the strict CPU feature diff --git a/clanker/plans/GPUTodo.md b/clanker/plans/GPUTodo.md index 47372d395..26c86d352 100644 --- a/clanker/plans/GPUTodo.md +++ b/clanker/plans/GPUTodo.md @@ -772,14 +772,14 @@ It is deliberately slow and never an automatic fallback. M2 is complete within this gated test-only scope. Production live integration and final on-screen validation remain open; `iris-gui` cannot select this mode. -The Windows gate `.\scripts\test-resident-gate.ps1` checks 28 resident CPU, -4 flat-fill, 2 line, 2 HostW walker, 1 HostW interpreter/JIT source oracle, -1 shade-DDA, 1 pattern-state, 3 command-ABI and 3 direct-state tests, plus 19 -resident GPU and 8 +The Windows gate `.\scripts\test-resident-gate.ps1` checks 103 cases: 28 +resident CPU, 4 flat-fill, 2 line, 1 shade-DDA, 1 pattern-state, 3 command-ABI +and 3 direct-state tests, 2 HostW walker tests, 1 HostW interpreter/JIT source +oracle, 2 resident HostR port/retirement tests, plus 19 resident GPU and 9 direct-scanout tests on each of Vulkan and D3D12. It requires the named backend on an integrated/discrete adapter and a native desktop, and fails rather than skipping when unavailable; a single-backend run is not the -complete 99-test gate. See +complete 103-test gate. See `TESTING.md` for commands and prerequisites. ### M3 - Command ABI and complete uber-shader @@ -988,36 +988,71 @@ no per-qword GPU submission. HOSTR remains M5 work. ### M5 - HOSTR and CPU observation -- [ ] Add a fixed reusable full-plane readback path. -- [ ] Synchronize only the selected source plane(s). -- [ ] Run the existing CPU HOSTR walker against the synchronized snapshot. -- [ ] Reuse one snapshot for the full PIO/DMA transfer generation. -- [ ] Preserve PIO read-then-advance and DMA prime-then-read behavior. -- [ ] Reuse the synchronization primitive for snapshots and monitor raw-plane +- [x] Add a fixed reusable full-plane readback path. +- [x] Synchronize only the selected source plane(s). +- [x] Run the existing CPU HOSTR walker against the synchronized snapshot. +- [x] Reuse one snapshot for the full PIO/DMA transfer generation. +- [x] Preserve PIO read-then-advance and DMA prime-then-read behavior. +- [x] Reuse the synchronization primitive for snapshots and monitor raw-plane reads. -- [ ] Measure before considering region readback. - -Complete when HOSTR matches CPU mode and performs at most one plane readback -per transfer generation, never one per qword. - -The test-only resident backend already has reusable, explicitly selected raw -plane observation and shadow acknowledgement; none of these checkboxes imply -that the board's HOSTR, snapshot or monitor consumers are wired to it. +- [x] Measure before considering region readback. + +The resident raster keeps one bounded full-plane CPU snapshot for the selected +HostR source. Its cache key is the plane plus the resident epoch and last-write +position. A miss uses the existing typed observation path to submit the exact +accepted prefix, copy only RGB or auxiliary through the fixed staging buffer, +and attribute the complete-plane bytes to `HostRead`. The existing CPU HostR +walker then consumes that snapshot; it never receives `fb_rgb` or `fb_aux`. +PIO and bulk DMA GOs sharing an unchanged source generation reuse the same +snapshot, including across transfer boundaries. A selected-plane write, +restore or reset changes the key and forces exactly one new observation. + +The board ports retain their architectural phase: PIO reads return the current +latch before a GO advances it, scalar DMA primes before reading, and a bulk DMA +token fills and retires its complete HostRW run once. READ advances the +resident stream without changing a plane generation. CPU port tests pin these +phases and sticky failure, while the Vulkan/D3D12 hardware oracle reads known +GPU-written words through PIO and DMA, proves one map for repeated reads, +forces a second map after a selected-plane write, and verifies that the board +CPU plane remains stale. + +M5 is complete within the gated resident backend: HostR matches CPU mode and +performs at most one selected-plane readback per transfer generation, never +one per qword. Snapshot and monitor board consumers remain later explicit +observer/lifecycle work; they already share the same attributed observation +primitive. ### M6 - Explicit observers and frontend matrix -- [ ] Read back composed output for screenshots and VNC/CI capture. -- [ ] Keep raw-plane reads separate from composed-frame reads. -- [ ] Validate CPU+GL, CPU+wgpu, wgpu+wgpu and readback wgpu+GL/CPU modes. -- [ ] Label the readback compositor mode as compatibility/validation only. -- [ ] Ensure capture users cannot silently force the proper GPU path into +- [x] Read back composed output for screenshots and VNC/CI capture. +- [x] Keep raw-plane reads separate from composed-frame reads. +- [x] Validate CPU+GL, CPU+wgpu, wgpu+wgpu and readback wgpu+GL/CPU modes. +- [x] Label the readback compositor mode as compatibility/validation only. +- [x] Ensure capture users cannot silently force the proper GPU path into continuous raw-plane readback without reporting the mode change. Complete when all four combinations produce matching images and only the documented combinations/readers increment readback counters. -Composed-output screenshot, headless and VNC capture are exercised by the -test-only resident display; live frontend integration remains open. +Complete within the gated resident backend. The test-only display has a fixed, +typed `DirectWgpu` or `ReadbackCompatibility` mode. Renderer rejection, +capture demand and capture failure cannot change that selection. Direct +presentation and its screenshot, frame-request, headless and VNC consumers +read back only the composed output; they produce no raw-plane observation. +Compatibility presentation performs an attributed two-plane +`Compatibility` observation, then uses a separate CPU cache and the ordinary +renderer. Per-mode composed-capture attempts, successes and bytes are counted +separately from typed raw-plane observation traffic and survive display +shutdown and resident reset. + +The dual-backend matrix compares CPU software/GL composition, CPU planes +through wgpu scanout, resident planes through same-device wgpu scanout and +resident readback through the ordinary CPU renderer. Capture tickets are +coalesced at the frame cut, newer tickets remain pending, and failures neither +publish stale pixels nor advance `frame_seq`. The complete screenshot, +headless frame and loopback VNC cases run in both resident modes. Live +frontend integration remains open; no production selector exposes either +resident mode yet. ### M7 - Snapshot, reset and failure lifecycle diff --git a/scripts/test-resident-gate.ps1 b/scripts/test-resident-gate.ps1 index 0166f2072..8367c85ac 100644 --- a/scripts/test-resident-gate.ps1 +++ b/scripts/test-resident-gate.ps1 @@ -71,6 +71,7 @@ try { Invoke-ResidentTests 'dev::ng1::wgpu::line::tests' 2 $false Invoke-ResidentTests 'dev::ng1::wgpu::host::tests' 2 $false Invoke-ResidentTests 'mixed_host_sources_match_between_interpreter_and_jit' 1 $false + Invoke-ResidentTests 'test_hostr_board_' 2 $false Invoke-ResidentTests 'dev::ng1::wgpu::shade::tests' 1 $false Invoke-ResidentTests 'line_pattern_closed_form_matches_scalar_advancement' 1 $false Invoke-ResidentTests 'dev::ng1::wgpu::command::tests' 3 $false @@ -78,7 +79,7 @@ try { foreach ($name in $requestedBackends) { $env:REX3_RESIDENT_BACKEND = $name Invoke-ResidentTests 'resident_gpu_' 19 $true - Invoke-ResidentTests 'dev::ng1::wgpu::direct_tests' 8 $true + Invoke-ResidentTests 'dev::ng1::wgpu::direct_tests' 9 $true } } finally { [Environment]::SetEnvironmentVariable('REX3_RESIDENT_BACKEND', $previousBackend, 'Process') diff --git a/src/dev/ng1/rex3.rs b/src/dev/ng1/rex3.rs index c7aec8b34..ea2c10497 100644 --- a/src/dev/ng1/rex3.rs +++ b/src/dev/ng1/rex3.rs @@ -1859,8 +1859,10 @@ impl Rex3 { RasterMode::Resident(raster) => { let host_len = context.host_len; let host_cursor = context.host_cursor; - let resident_owns_host = context.drawmode0.opcode() == DRAWMODE0_OPCODE_DRAW - && (context.drawmode0.colorhost() || context.drawmode0.alphahost()); + let opcode = context.drawmode0.opcode(); + let resident_owns_host = opcode == DRAWMODE0_OPCODE_READ + || (opcode == DRAWMODE0_OPCODE_DRAW + && (context.drawmode0.colorhost() || context.drawmode0.alphahost())); let mut result = raster.enqueue(context); if result.is_ok() && !resident_owns_host && host_len > 1 && host_cursor < host_len { // M3 consumes no host words, but execute_go still retries one @@ -1967,21 +1969,8 @@ impl Rex3 { if self.raster.lock().kind() != RasterModeKind::Resident { return Err(ResidentError::InvalidState("resident raster is not active")); } - let resident_gpu = match &*self.raster.lock() { - RasterMode::Resident(raster) => raster.gpu(), - _ => None, - }; - if let Some(renderer) = self.head.renderer_slot().lock().as_deref() { - if renderer.takes_gpu_frames() { - let gpu = resident_gpu.as_ref() - .ok_or(ResidentError::Unsupported("resident renderer requires a GPU"))?; - frame::validate_presenter(renderer, gpu)?; - } else if !renderer.takes_resident_readback_frames() { - return Err(ResidentError::Unsupported( - "resident frames require a wgpu presenter or labeled readback renderer", - )); - } - } + // The processor does not select a display; the explicitly chosen + // display mode validates its renderer when it starts and renders. let mut processor = self.processor_thread.lock(); if let Some(error) = self.resident_fault() { return Err(error); @@ -3228,6 +3217,33 @@ impl Rex3 { self.flush_gpu_replay(FlushReason::WaitIdle); } + // A resident DMA read can return BUS_BUSY only before publishing its GO. + fn queue_dma_host_read(&self, token: u32, count: u64) -> u32 { + #[cfg(all(test, feature = "wgpu"))] + { + let mode = self.raster.lock(); + if let RasterMode::Resident(raster) = &*mode { + if raster.fault().is_some() || !self.running.load(Ordering::Acquire) { + return BUS_ERR; + } + drop(mode); + return if self.gfifo_try_push(token, count) { BUS_OK } else { BUS_BUSY }; + } + } + self.gfifo_push(token, count); + BUS_OK + } + + // Unlike wait_idle, a resident failure must reach the bus rather than panic. + fn wait_dma_host_read(&self) -> bool { + #[cfg(all(test, feature = "wgpu"))] + if self.raster.lock().kind() == RasterModeKind::Resident { + return self.wait_resident_idle().is_ok(); + } + self.wait_idle(); + true + } + fn register_processor(&self) { // Publish our thread handle so gfifo_push can unpark us when we park. let _ = self.processor_unparker.set(thread::current()); @@ -4850,9 +4866,10 @@ impl Rex3 { let offset = addr & (REX3_SIZE - 1); let reg_offset = offset & !0x0800; let is_go = offset & 0x0800 != 0; + // A failed worker may leave the FIFO permanently busy. #[cfg(all(test, feature = "wgpu"))] if matches!(reg_offset, REX3_HOSTRW0 | REX3_HOSTRW1) - && self.reject_resident_operation("HOSTRW reads").is_err() + && self.resident_fatal_notified.load(Ordering::Acquire) { return BusRead32::err(); } @@ -5017,6 +5034,11 @@ impl Rex3 { } }; + #[cfg(all(test, feature = "wgpu"))] + if matches!(reg_offset, REX3_HOSTRW0 | REX3_HOSTRW1) && !self.resident_ready() { + return BusRead32::err(); + } + // STATUS is polled constantly — suppress from debug log. if reg_offset != REX3_STATUS && reg_offset != REX3_USER_STATUS { if result.is_ok() { @@ -5187,7 +5209,7 @@ impl Rex3 { let reg_offset64r = offset & !0x0800; #[cfg(all(test, feature = "wgpu"))] if reg_offset64r == REX3_HOSTRW0 - && self.reject_resident_operation("HOSTRW reads").is_err() + && self.resident_fatal_notified.load(Ordering::Acquire) { return BusRead64::err(); } @@ -5199,6 +5221,10 @@ impl Rex3 { if self.gfxbusy.load(Ordering::Acquire) || !self.gfifo.is_empty() { return BusRead64::busy(); } + #[cfg(all(test, feature = "wgpu"))] + if !self.resident_ready() { + return BusRead64::err(); + } let val = unsafe { (*self.context.get()).hostrw_get() }; self.note_hostrw_read(); #[cfg(feature = "developer")] @@ -5237,20 +5263,19 @@ impl Rex3 { let offset = addr & (REX3_SIZE - 1); let is_go64r = (offset & 0x0800) != 0; let reg_offset64r = offset & !0x0800; - #[cfg(all(test, feature = "wgpu"))] - if reg_offset64r == REX3_HOSTRW0 - && self.reject_resident_operation("HOSTRW reads").is_err() - { - return BusRead64::err(); - } if reg_offset64r != REX3_HOSTRW0 || !is_go64r { return self.bus_read64(addr); } #[cfg(feature = "developer")] let before = unsafe { (*self.context.get()).hostrw_get() }; - self.gfifo_push(REX3_DMA_PURE_GO, 0); - self.wait_idle(); + let status = self.queue_dma_host_read(REX3_DMA_PURE_GO, 0); + if status != BUS_OK { + return BusRead64 { status, data: 0 }; + } + if !self.wait_dma_host_read() { + return BusRead64::err(); + } let val = unsafe { (*self.context.get()).hostrw_get() }; self.note_hostrw_read(); @@ -5314,19 +5339,20 @@ impl Rex3 { return BUS_ERR; } if out.is_empty() { return BUS_OK; } - #[cfg(all(test, feature = "wgpu"))] - if self.reject_resident_operation("HOSTRW reads").is_err() { - return BUS_ERR; - } if out.len() > HOSTRW_BUF_QWORDS { return BUS_ERR; } - self.gfifo_push(REX3_DMA_BATCH_R, out.len() as u64); - self.wait_idle(); + let status = self.queue_dma_host_read(REX3_DMA_BATCH_R, out.len() as u64); + if status != BUS_OK { + return status; + } + if !self.wait_dma_host_read() { + return BUS_ERR; + } - // SAFETY: wait_idle() means the consumer has finished and its writes to - // ctx.hostrw are visible (gfxbusy is loaded Acquire). + // SAFETY: the successful wait means the consumer has finished and its + // writes to ctx.hostrw are visible (gfxbusy is loaded Acquire). unsafe { let ctx = &*self.context.get(); for (i, slot) in out.iter_mut().enumerate() { diff --git a/src/dev/ng1/rex3_frame.rs b/src/dev/ng1/rex3_frame.rs index 374bca2a2..e6bf97e36 100644 --- a/src/dev/ng1/rex3_frame.rs +++ b/src/dev/ng1/rex3_frame.rs @@ -7,17 +7,33 @@ use std::time::{Duration, Instant}; use parking_lot::{Condvar, Mutex, MutexGuard}; use super::{RasterMode, Rex3, Renderer, GFIFO_RESIDENT_FRAME, STATUS_VRINT}; -use crate::dev::ng1::wgpu::direct::{CaptureFault, Ng1ResidentScanout, ResidentFrameStamp, ResidentScanoutTraffic}; +use crate::compositor::SwCompositor; +use crate::dev::ng1::wgpu::direct::{ + CaptureCount, CaptureFault, Ng1ResidentScanout, ResidentFrameStamp, ResidentScanoutTraffic, +}; use crate::dev::ng1::wgpu::resident::{ Observation, ObservationReason, PlaneMask, Position, ResidentError, }; use crate::dev::ng1::wgpu::Ng1Scanout; -use crate::disp::{BarStats, StatusBar, StatusBarTexture, FB_STRIDE}; +use crate::disp::{BarStats, Rex3Screen, StatusBar, StatusBarTexture, FB_STRIDE}; use crate::debug_overlay::DebugOverlay; use crate::wgpu::Gpu; const FRAME_WAIT: Duration = Duration::from_millis(50); +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub(crate) enum ResidentDisplayMode { + DirectWgpu, + ReadbackCompatibility, +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub(crate) struct ResidentDisplayFrame { + pub(crate) position: Position, + pub(crate) width: u32, + pub(crate) height: u32, +} + struct Marker { id: u64, reached: Option>, @@ -29,6 +45,21 @@ struct MarkerState { active: Option, } +#[derive(Default)] +struct CaptureTotals { + direct: CaptureCount, + compatibility: CaptureCount, +} + +impl CaptureTotals { + fn for_mode(&mut self, mode: ResidentDisplayMode) -> &mut CaptureCount { + match mode { + ResidentDisplayMode::DirectWgpu => &mut self.direct, + ResidentDisplayMode::ReadbackCompatibility => &mut self.compatibility, + } + } +} + /// One marker at a time. The processor acknowledges the marker and then /// waits without holding the raster lock until the display releases it. #[derive(Default)] @@ -42,6 +73,8 @@ pub(super) struct ResidentFrameControl { screenshots: AtomicU64, screenshots_completed: AtomicU64, captured: AtomicBool, + // Like raw observations, composed-capture totals survive resident resets. + captures: Mutex, error: Mutex>, } @@ -109,6 +142,27 @@ impl ResidentFrameControl { self.captured.store(false, Ordering::Release); } + fn capture_count(&self, mode: ResidentDisplayMode) -> CaptureCount { + *self.captures.lock().for_mode(mode) + } + + fn capture_attempt(&self, mode: ResidentDisplayMode, bytes: u64) { + self.captures.lock().for_mode(mode).attempt(bytes); + } + + fn capture_success(&self, mode: ResidentDisplayMode, bytes: u64) { + self.captures.lock().for_mode(mode).success(bytes); + } + + fn capture_delta(&self, mode: ResidentDisplayMode, before: CaptureCount, after: CaptureCount) { + let mut totals = self.captures.lock(); + let total = totals.for_mode(mode); + total.attempts += after.attempts - before.attempts; + total.attempted_bytes += after.attempted_bytes - before.attempted_bytes; + total.successes += after.successes - before.successes; + total.successful_bytes += after.successful_bytes - before.successful_bytes; + } + pub(super) fn wake(&self) { self.changed.notify_all(); } @@ -138,6 +192,10 @@ impl Drop for FrameCut<'_> { } impl Rex3 { + pub(crate) fn resident_composed_captures(&self, mode: ResidentDisplayMode) -> CaptureCount { + self.resident_frame.capture_count(mode) + } + fn resident_display_fence_complete(&self) -> bool { let target = self.xmap_fence.load(Ordering::Acquire); self.gfifo_fence.load(Ordering::Acquire).wrapping_sub(target) < 0x8000_0000 @@ -255,7 +313,11 @@ impl Rex3 { } /// Opt in only from tests, after starting the resident GFIFO processor. - pub(crate) fn start_resident_display(&self, gpu: Arc) -> Result<(), ResidentError> { + pub(crate) fn start_resident_display( + &self, + gpu: Arc, + mode: ResidentDisplayMode, + ) -> Result<(), ResidentError> { if !self.running.load(Ordering::Acquire) { return Err(ResidentError::InvalidState("resident GFIFO is stopped")); } @@ -263,7 +325,7 @@ impl Rex3 { if thread_slot.is_some() { return Err(ResidentError::InvalidState("resident display already started")); } - let mut display = ResidentDisplay::new(self, &gpu)?; + let mut display = ResidentDisplay::new(self, &gpu, mode)?; if !self.running.load(Ordering::Acquire) { return Err(ResidentError::InvalidState("resident GFIFO is stopped")); } @@ -308,6 +370,30 @@ pub(super) fn validate_presenter(renderer: &dyn Renderer, gpu: &Arc) -> Res Ok(()) } +fn validate_compatibility_presenter(renderer: &dyn Renderer) -> Result<(), ResidentError> { + if !renderer.takes_resident_readback_frames() { + return Err(ResidentError::Unsupported("resident compatibility display requires a readback-capable renderer")); + } + Ok(()) +} + +fn publish_capture(rex: &Rex3, screen: &mut Rex3Screen, requests: Requests, save_screenshots: bool) { + screen.frame_seq = screen.frame_seq.wrapping_add(1); + rex.resident_frame.complete(requests); + if requests.needs_screenshot && save_screenshots { + let (width, height) = (screen.width, screen.height); + let pixels = screen.rgba.clone(); + let n = rex.screenshot_counter.fetch_add(1, Ordering::Relaxed); + let path = format!("screenshot_{n:04}.png"); + std::thread::spawn(move || { + match crate::disp::save_screenshot(&path, &pixels, width, height) { + Ok(()) => println!("iris: screenshot saved to {path}"), + Err(error) => println!("iris: screenshot failed: {error}"), + } + }); + } +} + fn observed_words(observation: &Observation, rgb: bool) -> Result, ResidentError> { let bytes = if rgb { observation.rgb() } else { observation.aux() } .ok_or(ResidentError::InvalidState("compatibility observation is missing a plane"))?; @@ -324,60 +410,81 @@ fn observed_words(observation: &Observation, rgb: bool) -> Result, Resi /// Explicitly slow validation/compatibility path: read both authoritative /// planes, then hand the CPU snapshots to the ordinary renderer. -pub(crate) struct ResidentCompatibilityDisplay { +struct ResidentCompatibilityDisplay { overlay: DebugOverlay, status: StatusBar, sbtex: StatusBarTexture, + compositor: Option, + capture: CaptureCount, + capture_fault: Option, + save_screenshots: bool, resize_pending: bool, } impl ResidentCompatibilityDisplay { - pub(crate) fn new(rex: &Rex3) -> Result { - if !rex.head.renderer_slot().lock().as_deref() - .is_some_and(Renderer::takes_resident_readback_frames) - { - return Err(ResidentError::Unsupported("resident compatibility display requires a renderer")); + fn new(rex: &Rex3) -> Result { + if let Some(renderer) = rex.head.renderer_slot().lock().as_deref() { + validate_compatibility_presenter(renderer)?; } Ok(Self { overlay: DebugOverlay::new(), status: StatusBar::new(), sbtex: StatusBarTexture::new(), + compositor: None, + capture: CaptureCount::default(), + capture_fault: None, + save_screenshots: true, resize_pending: true, }) } - pub(crate) fn render( + fn render( &mut self, rex: &Rex3, timeout: Duration, at_marker: impl FnOnce(Position), - ) -> Result { + ) -> Result { rex.wait_resident_display_fence(timeout)?; let cut = rex.resident_frame_cut(timeout)?; + let requests = cut.requests; + let capture = requests.needs_frame || requests.needs_screenshot + || rex.head.capturing(); + if capture { + rex.resident_frame.invalidate(); + } at_marker(cut.position); let mut screen = rex.head.screen().lock(); - rex.resident_frame.invalidate(); screen.topscan = cut.topscan; screen.status_bar_only = false; - self.resize_pending |= screen.refresh_display_state( + let resized = screen.refresh_display_state( &rex.vc2, &rex.xmap0, &rex.cmap0, &rex.bt445, &rex.diag, ); + if resized { + rex.resident_frame.invalidate(); + } + self.resize_pending |= resized; if !rex.resident_display_fence_complete() { return Err(ResidentError::Timeout("display fence after frame marker")); } + if screen.width == 0 || screen.width > FB_STRIDE || screen.height == 0 + || screen.height > super::REX3_SCREEN_HEIGHT as usize + { + return Err(ResidentError::InvalidState("resident scanout dimensions exceed the Newport planes")); + } self.overlay.show_cmap = rex.show_cmap.load(Ordering::Relaxed); self.overlay.show_disp_debug = rex.show_disp_debug.load(Ordering::Relaxed); self.overlay.show_draw_debug = rex.draw_debug_active(); let stats: BarStats = rex.status.snapshot(Instant::now(), rex.gfifo.len()); let mut renderer = rex.head.renderer_slot().lock(); - let renderer = renderer.as_deref_mut() - .ok_or(ResidentError::Unsupported("resident compatibility display requires a renderer"))?; + if let Some(renderer) = renderer.as_deref() { + validate_compatibility_presenter(renderer)?; + } let mut mode = rex.raster.lock(); let RasterMode::Resident(raster) = &mut *mode else { return Err(ResidentError::InvalidState("resident raster is not active")); @@ -401,7 +508,7 @@ impl ResidentCompatibilityDisplay { damage.mark_full(DamagePlane::Aux); damage }; - self.resize_pending |= screen.refresh( + let resized = screen.refresh( &rgb, &aux, #[cfg(feature = "rex-dirty-upload")] @@ -414,26 +521,60 @@ impl ResidentCompatibilityDisplay { &rex.bt445, &rex.diag, ); - if self.resize_pending { - renderer.resize(screen.width, screen.height); - self.resize_pending = false; - } - renderer.present( - &mut screen, - &mut self.overlay, - &mut self.status, - &mut self.sbtex, - &stats, - false, - None, - None, - ); - Ok(cut.position) + if resized { + rex.resident_frame.invalidate(); + } + self.resize_pending |= resized; + if !rex.resident_display_fence_complete() { + return Err(ResidentError::Timeout("display fence after compatibility observation")); + } + if let Some(renderer) = renderer.as_deref_mut() { + if self.resize_pending { + renderer.resize(screen.width, screen.height); + self.resize_pending = false; + } + renderer.present( + &mut screen, + &mut self.overlay, + &mut self.status, + &mut self.sbtex, + &stats, + false, + None, + None, + ); + } else { + self.resize_pending = true; + } + let (width, height) = (screen.width, screen.height); + if capture { + let bytes = (width * height * size_of::()) as u64; + self.capture.attempt(bytes); + rex.resident_frame.capture_attempt(ResidentDisplayMode::ReadbackCompatibility, bytes); + let fault = self.capture_fault.take(); + if matches!(fault, Some(CaptureFault::BeforeCapture)) { + return Err(ResidentError::Failed("injected composed capture failure before capture".into())); + } + let compositor = self.compositor.get_or_insert_with(SwCompositor::new); + compositor.compose_pixels(&screen.compositor_source()); + if matches!(fault, Some(CaptureFault::AfterStaging)) { + return Err(ResidentError::Failed("injected composed capture failure after staging".into())); + } + for y in 0..height { + let row = y * FB_STRIDE; + screen.rgba[row..row + width] + .copy_from_slice(&compositor.pixels()[row..row + width]); + } + self.capture.success(bytes); + rex.resident_frame.capture_success(ResidentDisplayMode::ReadbackCompatibility, bytes); + publish_capture(rex, &mut screen, requests, self.save_screenshots); + } + Ok(ResidentDisplayFrame { position: cut.position, width: width as u32, height: height as u32 }) } } /// Owns the composed output and its staging buffer across refreshes. -pub(crate) struct ResidentDisplay { +struct ResidentDirectDisplay { scanout: Ng1ResidentScanout, overlay: DebugOverlay, status: StatusBar, @@ -443,8 +584,8 @@ pub(crate) struct ResidentDisplay { resize_pending: bool, } -impl ResidentDisplay { - pub(crate) fn new(rex: &Rex3, gpu: &Arc) -> Result { +impl ResidentDirectDisplay { + fn new(rex: &Rex3, gpu: &Arc) -> Result { if let Some(renderer) = rex.head.renderer_slot().lock().as_deref() { validate_presenter(renderer, gpu)?; } @@ -460,21 +601,9 @@ impl ResidentDisplay { }) } - pub(crate) fn traffic(&self) -> ResidentScanoutTraffic { - self.scanout.traffic() - } - - pub(crate) fn fail_next_capture(&mut self, fault: CaptureFault) { - self.scanout.fail_next_capture(fault); - } - - pub(crate) fn suppress_screenshot_files(&mut self) { - self.save_screenshots = false; - } - /// The hook runs at the FIFO cut, before the raster lock is taken. It is /// used to deterministically queue later draws in the ordering tests. - pub(crate) fn render( + fn render( &mut self, rex: &Rex3, timeout: Duration, @@ -485,10 +614,12 @@ impl ResidentDisplay { let requests = cut.requests; let capture = requests.needs_frame || requests.needs_screenshot || rex.head.capturing(); + if capture { + rex.resident_frame.invalidate(); + } at_marker(cut.position); let mut screen = rex.head.screen().lock(); - rex.resident_frame.invalidate(); screen.topscan = cut.topscan; screen.status_bar_only = false; let resized = screen.refresh_resident_display_state( @@ -498,6 +629,9 @@ impl ResidentDisplay { &rex.bt445, &rex.diag, ); + if resized { + rex.resident_frame.invalidate(); + } self.resize_pending |= resized; // A mode write after the initial wait may have published a later // fence. Its GFIFO entry cannot pass this marker, so defer rather @@ -553,7 +687,11 @@ impl ResidentDisplay { if capture { let (width, height) = (stamp.width as usize, stamp.height as usize); self.capture.resize(width * height, 0); - self.scanout.capture(stamp, raster, &mut self.capture)?; + let before = self.scanout.traffic().composed_readbacks; + let result = self.scanout.capture(stamp, raster, &mut self.capture); + let after = self.scanout.traffic().composed_readbacks; + rex.resident_frame.capture_delta(ResidentDisplayMode::DirectWgpu, before, after); + result?; for y in 0..height { let row = y * FB_STRIDE; for (dst, &src) in screen.rgba[row..row + width] @@ -562,22 +700,100 @@ impl ResidentDisplay { *dst = 0xff00_0000 | src.swap_bytes() >> 8; } } - screen.frame_seq = screen.frame_seq.wrapping_add(1); - rex.resident_frame.complete(requests); - if requests.needs_screenshot && self.save_screenshots { - let pixels = screen.rgba.clone(); - let n = rex.screenshot_counter.fetch_add(1, Ordering::Relaxed); - let path = format!("screenshot_{n:04}.png"); - std::thread::spawn(move || { - match crate::disp::save_screenshot(&path, &pixels, width, height) { - Ok(()) => println!("iris: screenshot saved to {path}"), - Err(error) => println!("iris: screenshot failed: {error}"), - } - }); - } + publish_capture(rex, &mut screen, requests, self.save_screenshots); } Ok(stamp) } +} + +enum ResidentDisplayDriver { + Direct(ResidentDirectDisplay), + Compatibility(ResidentCompatibilityDisplay), +} + +/// The selected mode is fixed for this display; errors never change it. +pub(crate) struct ResidentDisplay { + gpu: Arc, + driver: ResidentDisplayDriver, +} + +impl ResidentDisplay { + pub(crate) fn new(rex: &Rex3, gpu: &Arc, mode: ResidentDisplayMode) -> Result { + let resident_gpu = { + let mut raster = rex.raster.lock(); + let RasterMode::Resident(raster) = &mut *raster else { + return Err(ResidentError::InvalidState("resident raster is not active")); + }; + raster.status()?; + raster.gpu().ok_or(ResidentError::Unsupported("resident display requires a GPU"))? + }; + if !Arc::ptr_eq(&resident_gpu, gpu) { + return Err(ResidentError::DeviceMismatch); + } + let driver = match mode { + ResidentDisplayMode::DirectWgpu => ResidentDisplayDriver::Direct(ResidentDirectDisplay::new(rex, gpu)?), + ResidentDisplayMode::ReadbackCompatibility => { + ResidentDisplayDriver::Compatibility(ResidentCompatibilityDisplay::new(rex)?) + } + }; + Ok(Self { gpu: Arc::clone(gpu), driver }) + } + + pub(crate) fn mode(&self) -> ResidentDisplayMode { + match self.driver { + ResidentDisplayDriver::Direct(_) => ResidentDisplayMode::DirectWgpu, + ResidentDisplayDriver::Compatibility(_) => ResidentDisplayMode::ReadbackCompatibility, + } + } + + pub(crate) fn direct_traffic(&self) -> Option { + match &self.driver { + ResidentDisplayDriver::Direct(display) => Some(display.scanout.traffic()), + ResidentDisplayDriver::Compatibility(_) => None, + } + } + + pub(crate) fn capture_count(&self) -> CaptureCount { + match &self.driver { + ResidentDisplayDriver::Direct(display) => display.scanout.traffic().composed_readbacks, + ResidentDisplayDriver::Compatibility(display) => display.capture, + } + } + + pub(crate) fn fail_next_capture(&mut self, fault: CaptureFault) { + match &mut self.driver { + ResidentDisplayDriver::Direct(display) => display.scanout.fail_next_capture(fault), + ResidentDisplayDriver::Compatibility(display) => display.capture_fault = Some(fault), + } + } + + pub(crate) fn suppress_screenshot_files(&mut self) { + match &mut self.driver { + ResidentDisplayDriver::Direct(display) => display.save_screenshots = false, + ResidentDisplayDriver::Compatibility(display) => display.save_screenshots = false, + } + } + + pub(crate) fn render( + &mut self, + rex: &Rex3, + timeout: Duration, + at_marker: impl FnOnce(Position), + ) -> Result { + let requests = rex.resident_frame.requests(); + // Retain a completed capture across uncaptured presentations; otherwise + // a CI consumer can miss it before it acquires the screen lock. + if requests.needs_frame || requests.needs_screenshot || rex.head.capturing() { + rex.resident_frame.invalidate(); + } + match &mut self.driver { + ResidentDisplayDriver::Direct(display) => { + let stamp = display.render(rex, timeout, at_marker)?; + Ok(ResidentDisplayFrame { position: stamp.position, width: stamp.width, height: stamp.height }) + } + ResidentDisplayDriver::Compatibility(display) => display.render(rex, timeout, at_marker), + } + } fn run(&mut self, rex: &Rex3) { let period = Duration::from_micros(16667); @@ -591,7 +807,7 @@ impl ResidentDisplay { *rex.resident_frame.error.lock() = result.as_ref().err().cloned(); if let Err(error) = result { let fatal = rex.resident_fault().or_else(|| { - self.scanout.gpu().check().err().map(|message| { + self.gpu.check().err().map(|message| { if message.starts_with("wgpu device lost (") { ResidentError::DeviceLost(message) } else { @@ -652,6 +868,17 @@ mod tests { assert!(control.frame_available()); control.invalidate(); assert!(!control.frame_available(), "an unpublished display state is not a captured frame"); + + control.capture_attempt(ResidentDisplayMode::DirectWgpu, 64); + control.capture_success(ResidentDisplayMode::DirectWgpu, 64); + control.capture_attempt(ResidentDisplayMode::ReadbackCompatibility, 32); + control.activate(); + assert_eq!(control.capture_count(ResidentDisplayMode::DirectWgpu), CaptureCount { + attempts: 1, successes: 1, attempted_bytes: 64, successful_bytes: 64, + }); + assert_eq!(control.capture_count(ResidentDisplayMode::ReadbackCompatibility), CaptureCount { + attempts: 1, successes: 0, attempted_bytes: 32, successful_bytes: 0, + }); } #[test] diff --git a/src/dev/ng1/rex3_tests.rs b/src/dev/ng1/rex3_tests.rs index 4ba56204f..3e9f74609 100644 --- a/src/dev/ng1/rex3_tests.rs +++ b/src/dev/ng1/rex3_tests.rs @@ -375,13 +375,7 @@ mod resident_gate_tests { assert!(rex.load_framebuffers_named(path, "unused").is_err()); assert!(rex.dump_framebuffer_debug(path).is_err()); assert!(rex.load_state(&toml::Value::Table(Default::default())).is_err()); - assert_eq!(rex.read32(set_addr(REX3_HOSTRW0)).status, BUS_ERR); - assert_eq!(rex.read32(go_addr(REX3_HOSTRW1)).status, BUS_ERR); - assert_eq!(rex.read64(set_addr(REX3_HOSTRW0)).status, BUS_ERR); - assert_eq!(rex.dma_read64(go_addr(REX3_HOSTRW0)).status, BUS_ERR); - let mut words = [0x1234; 2]; - assert_eq!(rex.dma_read64_bulk(go_addr(REX3_HOSTRW0), &mut words), BUS_ERR); - assert_eq!(words, [0x1234; 2]); + assert!(rex.execute_command("rex", &["fbdump"], Box::new(std::io::sink())).is_err()); assert!(rex.gfifo.is_empty()); let rejected = |operation: &dyn Fn()| { @@ -507,6 +501,12 @@ mod resident_gate_tests { queued += 1; } assert!(queued >= GFIFO_DEPTH - 2, "the failed worker leaves a full queue"); + assert_eq!(rex.read32(go_addr(REX3_HOSTRW0)).status, BUS_ERR); + assert_eq!(rex.read64(go_addr(REX3_HOSTRW0)).status, BUS_ERR); + assert_eq!(rex.dma_read64(go_addr(REX3_HOSTRW0)).status, BUS_ERR); + let mut words = [0x1234; 2]; + assert_eq!(rex.dma_read64_bulk(go_addr(REX3_HOSTRW0), &mut words), BUS_ERR); + assert_eq!(words, [0x1234; 2]); rex.stop(); assert!(!rex.is_running()); assert!(rex.processor_thread.lock().is_none()); @@ -762,6 +762,240 @@ mod resident_gate_tests { } } +#[cfg(feature = "wgpu")] +#[test] +fn test_hostr_board_ports_and_batch_retirement() { + use crate::dev::ng1::wgpu::resident::{ResidentBackend, ResidentError}; + use std::sync::atomic::AtomicUsize; + + struct PortProbe { + calls: Arc, + next: u64, + } + + impl ResidentBackend for PortProbe { + fn enqueue(&mut self, ctx: &mut Rex3Context) -> Result<(), ResidentError> { + assert_eq!(ctx.drawmode0.opcode(), DRAWMODE0_OPCODE_READ); + self.calls.fetch_add(1, Ordering::SeqCst); + for slot in &mut ctx.hostrw.0[..ctx.host_len as usize] { + *slot = self.next; + self.next += 1; + } + Ok(()) + } + + fn submit(&mut self) -> Result<(), ResidentError> { Ok(()) } + } + + with_big_stack(|| { + const LATCH: u64 = 0xa1b2_c3d4_e5f6_0718; + const FIRST: u64 = 0x1122_3344_5566_7701; + let calls = Arc::new(AtomicUsize::new(0)); + let mut rex = Box::new(Rex3::new(crate::disp::StatusSource::detached())); + let ctx = rex.context.get_mut(); + // The probe supplies synthetic port words; the unused address mode + // prevents the resident READ walker from replacing them. + ctx.drawmode0 = DrawMode0(DRAWMODE0_OPCODE_READ | (7 << DRAWMODE0_ADRMODE_SHIFT)); + ctx.drawmode1 = DrawMode1(DRAWMODE1_PLANES_NONE); + ctx.hostrw_set(LATCH); + let rgb = rex.fb_rgb.get_mut()[0]; + let aux = rex.fb_aux.get_mut()[0]; + rex.attach_resident_raster(PortProbe { calls: Arc::clone(&calls), next: FIRST }).unwrap(); + let rex = TestBoard::new(Arc::::from(rex)); + + rex.gfifo_push(REX3_WRMASK, 0); + assert_eq!(rex.read32(set_addr(REX3_HOSTRW0)).status, BUS_BUSY); + assert_eq!(rex.read32(go_addr(REX3_HOSTRW1)).status, BUS_BUSY); + assert_eq!(rex.read64(go_addr(REX3_HOSTRW0)).status, BUS_BUSY); + assert_eq!(rex.gfifo.len(), 1, "busy reads must not enqueue GO"); + let mut not_started = [LATCH; 2]; + assert_eq!(rex.dma_read64(go_addr(REX3_HOSTRW0)).status, BUS_ERR); + assert_eq!(rex.dma_read64_bulk(go_addr(REX3_HOSTRW0), &mut not_started), BUS_ERR); + assert_eq!(not_started, [LATCH; 2]); + assert_eq!(rex.gfifo.len(), 1); + rex.gfifo.reset(); + + // With no consumer, a full FIFO must refuse either DMA token without + // consuming the request; retrying it is then safe. + rex.running.store(true, Ordering::Release); + while rex.gfifo.try_push(REX3_WRMASK, 0) {} + let full = rex.gfifo.len(); + assert_eq!(rex.dma_read64(go_addr(REX3_HOSTRW0)).status, BUS_BUSY); + let mut refused = [LATCH; 2]; + assert_eq!(rex.dma_read64_bulk(go_addr(REX3_HOSTRW0), &mut refused), BUS_BUSY); + assert_eq!(refused, [LATCH; 2]); + assert_eq!(rex.gfifo.len(), full); + rex.running.store(false, Ordering::Release); + rex.gfifo.reset(); + + rex.start(); + let high = rex.read32(set_addr(REX3_HOSTRW0)); + let low = rex.read32(set_addr(REX3_HOSTRW1)); + let pio = rex.read64(set_addr(REX3_HOSTRW0)); + let dma = rex.dma_read64(set_addr(REX3_HOSTRW0)); + assert_eq!((high.status, high.data), (BUS_OK, (LATCH >> 32) as u32)); + assert_eq!((low.status, low.data), (BUS_OK, LATCH as u32)); + assert_eq!((pio.status, pio.data), (BUS_OK, LATCH)); + assert_eq!((dma.status, dma.data), (BUS_OK, LATCH)); + assert_eq!(calls.load(Ordering::SeqCst), 0, "SET reads must not issue GO"); + assert_eq!(rex.dma_read64_bulk(go_addr(REX3_HOSTRW0), &mut []), BUS_OK); + let mut invalid = [LATCH; 1]; + assert_eq!(rex.dma_read64_bulk(set_addr(REX3_HOSTRW0), &mut invalid), BUS_ERR); + assert_eq!(invalid, [LATCH; 1]); + + let high = rex.read32(go_addr(REX3_HOSTRW0)); + assert_eq!((high.status, high.data), (BUS_OK, (LATCH >> 32) as u32)); + rex.wait_resident_idle().unwrap(); + let low = rex.read32(go_addr(REX3_HOSTRW1)); + assert_eq!((low.status, low.data), (BUS_OK, FIRST as u32)); + rex.wait_resident_idle().unwrap(); + let pio = rex.read64(go_addr(REX3_HOSTRW0)); + assert_eq!((pio.status, pio.data), (BUS_OK, FIRST + 1)); + rex.wait_resident_idle().unwrap(); + assert_eq!(rex.read64(set_addr(REX3_HOSTRW0)).data, FIRST + 2); + + let dma = rex.dma_read64(go_addr(REX3_HOSTRW0)); + assert_eq!((dma.status, dma.data), (BUS_OK, FIRST + 3)); + assert_eq!(rex.dma_read64(set_addr(REX3_HOSTRW0)).data, FIRST + 3); + let mut words = [LATCH; 3]; + assert_eq!(rex.dma_read64_bulk(go_addr(REX3_HOSTRW0), &mut words), BUS_OK); + assert_eq!(words, [FIRST + 4, FIRST + 5, FIRST + 6]); + assert_eq!(calls.load(Ordering::SeqCst), 5, "a READ batch must execute only once"); + assert_eq!(rex.read64(set_addr(REX3_HOSTRW0)).data, FIRST + 4); + let mut next = [LATCH; 2]; + assert_eq!(rex.dma_read64_bulk(go_addr(REX3_HOSTRW0), &mut next), BUS_OK); + assert_eq!(next, [FIRST + 7, FIRST + 8]); + assert_eq!(calls.load(Ordering::SeqCst), 6, "each READ batch must execute only once"); + let ctx = unsafe { &*rex.context.get() }; + assert_eq!((ctx.host_len, ctx.host_cursor), (1, 0)); + assert_eq!(rex.read64(set_addr(REX3_HOSTRW0)).data, FIRST + 7); + assert_eq!(unsafe { (*rex.fb_rgb.get())[0] }, rgb); + assert_eq!(unsafe { (*rex.fb_aux.get())[0] }, aux); + assert_eq!(rex.jit_go_count.load(Ordering::Relaxed), 0); + assert_eq!(rex.interp_go_count.load(Ordering::Relaxed), 0); + #[cfg(feature = "rex-gpu")] + assert!(rex.gpu_replay_report().is_none()); + }); +} + +#[cfg(feature = "wgpu")] +#[test] +fn test_hostr_board_failures_are_sticky() { + use crate::dev::ng1::wgpu::resident::{ + Observation, ObservationReason, PlaneMask, ResidentBackend, ResidentError, ResidentStatus, + }; + use std::sync::atomic::AtomicUsize; + + struct ObservationFailure { + observations: Arc, + enqueues: Arc, + } + + impl ResidentBackend for ObservationFailure { + fn enqueue(&mut self, _: &mut Rex3Context) -> Result<(), ResidentError> { + self.enqueues.fetch_add(1, Ordering::SeqCst); + Ok(()) + } + fn submit(&mut self) -> Result<(), ResidentError> { Ok(()) } + fn status(&mut self) -> Result { Ok(ResidentStatus::for_test()) } + fn observe(&mut self, reason: ObservationReason, planes: PlaneMask) -> Result { + assert_eq!((reason, planes), (ObservationReason::HostRead, PlaneMask::Rgb)); + self.observations.fetch_add(1, Ordering::SeqCst); + Err(ResidentError::Failed("injected HostR observation failure".into())) + } + } + + struct SubmissionFailure(Arc); + + impl ResidentBackend for SubmissionFailure { + fn enqueue(&mut self, ctx: &mut Rex3Context) -> Result<(), ResidentError> { + self.0.fetch_add(1, Ordering::SeqCst); + ctx.hostrw[0] = 0x1234; + Ok(()) + } + fn submit(&mut self) -> Result<(), ResidentError> { + Err(ResidentError::Failed("injected HostR submission failure".into())) + } + } + + with_big_stack(|| { + const LATCH: u64 = 0xfeed_face_dead_beef; + for first in ["scalar DMA", "bulk DMA", "PIO"] { + let observations = Arc::new(AtomicUsize::new(0)); + let enqueues = Arc::new(AtomicUsize::new(0)); + let mut rex = Box::new(Rex3::new(crate::disp::StatusSource::detached())); + let ctx = rex.context.get_mut(); + ctx.drawmode0 = DrawMode0(DM0_READ_BLOCK); + ctx.drawmode1 = DrawMode1(DM1_RGB24_SRC); + ctx.hostrw_set(LATCH); + let rgb = rex.fb_rgb.get_mut()[0]; + let aux = rex.fb_aux.get_mut()[0]; + rex.attach_resident_raster(ObservationFailure { + observations: Arc::clone(&observations), + enqueues: Arc::clone(&enqueues), + }).unwrap(); + let rex = TestBoard::new(Arc::::from(rex)); + rex.start(); + + match first { + "scalar DMA" => assert_eq!(rex.dma_read64(go_addr(REX3_HOSTRW0)).status, BUS_ERR), + "bulk DMA" => { + let mut words = [LATCH; 2]; + assert_eq!(rex.dma_read64_bulk(go_addr(REX3_HOSTRW0), &mut words), BUS_ERR); + assert_eq!(words, [LATCH; 2]); + } + "PIO" => { + let read = rex.read64(go_addr(REX3_HOSTRW0)); + assert_eq!((read.status, read.data), (BUS_OK, LATCH)); + } + _ => unreachable!(), + } + let fault = ResidentError::Failed("injected HostR observation failure".into()); + assert_eq!(rex.wait_resident_idle(), Err(fault.clone())); + assert_eq!(rex.resident_fault(), Some(fault.clone())); + for offset in [REX3_HOSTRW0, REX3_HOSTRW1] { + assert_eq!(rex.read32(set_addr(offset)).status, BUS_ERR); + assert_eq!(rex.read32(go_addr(offset)).status, BUS_ERR); + } + assert_eq!(rex.read64(set_addr(REX3_HOSTRW0)).status, BUS_ERR); + assert_eq!(rex.read64(go_addr(REX3_HOSTRW0)).status, BUS_ERR); + assert_eq!(rex.dma_read64(set_addr(REX3_HOSTRW0)).status, BUS_ERR); + assert_eq!(rex.dma_read64(go_addr(REX3_HOSTRW0)).status, BUS_ERR); + let mut words = [LATCH; 2]; + assert_eq!(rex.dma_read64_bulk(go_addr(REX3_HOSTRW0), &mut words), BUS_ERR); + assert_eq!(words, [LATCH; 2]); + assert!(rex.gfifo.is_empty()); + assert_eq!(observations.load(Ordering::SeqCst), 1); + assert_eq!(enqueues.load(Ordering::SeqCst), 0); + assert_eq!(unsafe { (*rex.context.get()).hostrw[0] }, LATCH); + assert_eq!(unsafe { (*rex.fb_rgb.get())[0] }, rgb); + assert_eq!(unsafe { (*rex.fb_aux.get())[0] }, aux); + assert_eq!(rex.jit_go_count.load(Ordering::Relaxed), 0); + assert_eq!(rex.interp_go_count.load(Ordering::Relaxed), 0); + #[cfg(feature = "rex-gpu")] + assert!(rex.gpu_replay_report().is_none()); + } + + let calls = Arc::new(AtomicUsize::new(0)); + let mut rex = Box::new(Rex3::new(crate::disp::StatusSource::detached())); + let ctx = rex.context.get_mut(); + ctx.drawmode0 = DrawMode0(DRAWMODE0_OPCODE_READ | (7 << DRAWMODE0_ADRMODE_SHIFT)); + ctx.drawmode1 = DrawMode1(DRAWMODE1_PLANES_NONE); + ctx.hostrw_set(LATCH); + rex.attach_resident_raster(SubmissionFailure(Arc::clone(&calls))).unwrap(); + let rex = TestBoard::new(Arc::::from(rex)); + rex.start(); + let mut words = [LATCH; 2]; + assert_eq!(rex.dma_read64_bulk(go_addr(REX3_HOSTRW0), &mut words), BUS_ERR); + assert_eq!(words, [LATCH; 2], "failed submission must not expose generated words"); + let fault = ResidentError::Failed("injected HostR submission failure".into()); + assert_eq!(rex.resident_fault(), Some(fault.clone())); + assert_eq!(rex.dma_read64(go_addr(REX3_HOSTRW0)).status, BUS_ERR); + assert_eq!(rex.resident_fault(), Some(fault)); + assert_eq!(calls.load(Ordering::SeqCst), 1); + }); +} + /// A freshly reset, running Rex3. The order mirrors `Machine::reset`: /// power-on with the threads stopped, then start. /// diff --git a/src/dev/ng1/wgpu/direct.rs b/src/dev/ng1/wgpu/direct.rs index b63d7704d..0e5f5384a 100644 --- a/src/dev/ng1/wgpu/direct.rs +++ b/src/dev/ng1/wgpu/direct.rs @@ -97,6 +97,18 @@ pub(crate) struct CaptureCount { pub(crate) successful_bytes: u64, } +impl CaptureCount { + pub(crate) fn attempt(&mut self, bytes: u64) { + self.attempts += 1; + self.attempted_bytes += bytes; + } + + pub(crate) fn success(&mut self, bytes: u64) { + self.successes += 1; + self.successful_bytes += bytes; + } +} + #[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] pub(crate) struct DispatchCount { pub(crate) attempts: u64, @@ -179,8 +191,8 @@ impl Ng1ResidentFrame<'_> { #[derive(Clone, Copy)] pub(crate) enum CaptureFault { - BeforeCopy, - AfterMap, + BeforeCapture, + AfterStaging, } /// The group retains the raster's plane buffers without exposing their @@ -334,22 +346,20 @@ impl Ng1ResidentScanout { return Err(ResidentError::InvalidState("resident capture output is too small")); } let bytes = pixels as u64 * 4; - self.traffic.composed_readbacks.attempts += 1; - self.traffic.composed_readbacks.attempted_bytes += bytes; + self.traffic.composed_readbacks.attempt(bytes); let fault = self.capture_fault.take(); - if matches!(fault, Some(CaptureFault::BeforeCopy)) { - return Err(ResidentError::Failed("injected composed capture failure before copy".into())); + if matches!(fault, Some(CaptureFault::BeforeCapture)) { + return Err(ResidentError::Failed("injected composed capture failure before capture".into())); } self.capture_scratch.resize(pixels, 0); self.scanout.read_output(stamp.width, stamp.height, &mut self.capture_scratch) .map_err(scanout_error)?; - if matches!(fault, Some(CaptureFault::AfterMap)) { - return Err(ResidentError::Failed("injected composed capture failure after map".into())); + if matches!(fault, Some(CaptureFault::AfterStaging)) { + return Err(ResidentError::Failed("injected composed capture failure after staging".into())); } self.scanout.gpu().check().map_err(scanout_error)?; out[..pixels].copy_from_slice(&self.capture_scratch); - self.traffic.composed_readbacks.successes += 1; - self.traffic.composed_readbacks.successful_bytes += bytes; + self.traffic.composed_readbacks.success(bytes); Ok(()) } diff --git a/src/dev/ng1/wgpu/direct_tests.rs b/src/dev/ng1/wgpu/direct_tests.rs index 7d14d66e5..cc74f333c 100644 --- a/src/dev/ng1/wgpu/direct_tests.rs +++ b/src/dev/ng1/wgpu/direct_tests.rs @@ -25,7 +25,7 @@ use crate::dev::ng1::rex3::{ REX3_DCBDATA0, REX3_DCBMODE, REX3_DRAWMODE0, REX3_DRAWMODE1, REX3_TOPSCAN, REX3_WRMASK, REX3_XYENDI, REX3_XYSTARTI, }; -use crate::dev::ng1::rex3::frame::{ResidentCompatibilityDisplay, ResidentDisplay}; +use crate::dev::ng1::rex3::frame::{ResidentDisplay, ResidentDisplayMode}; use crate::dev::ng1::rex3_generic::{draw_with_fb, Framebuffers}; use crate::dev::ng1::rex3_shape; use crate::dev::ng1::vc2::{ @@ -1234,6 +1234,31 @@ fn assert_no_cpu_plane_access(rex: &Rex3) { assert_eq!((rgb.reads, rgb.writes, aux.reads, aux.writes), (0, 0, 0, 0)); } +fn assert_compatibility_observations(rex: &Rex3, frames: u64) { + let status = rex.resident_status().unwrap(); + let counts = status.observations.for_reason(ObservationReason::Compatibility); + let plane_bytes = PLANE_WORDS as u64 * 4; + assert_eq!((counts.attempts, counts.successes), (frames, frames)); + assert_eq!((counts.attempted_transfers, counts.successful_transfers), (frames * 2, frames * 2)); + assert_eq!((counts.attempted_bytes, counts.successful_bytes), (frames * 2 * plane_bytes, frames * 2 * plane_bytes)); + assert_eq!(status.raw_plane_map_attempts, frames); + for reason in [ + ObservationReason::HostRead, ObservationReason::Snapshot, ObservationReason::Monitor, + ObservationReason::Recording, ObservationReason::Validation, + ] { + assert_eq!(*status.observations.for_reason(reason), Default::default(), "{reason:?}"); + } +} + +fn assert_composed_captures(rex: &Rex3, display: &ResidentDisplay, attempts: u64, successes: u64) { + let bytes = (WIDTH * HEIGHT * 4) as u64; + let expected = super::direct::CaptureCount { + attempts, successes, attempted_bytes: attempts * bytes, successful_bytes: successes * bytes, + }; + assert_eq!(display.capture_count(), expected); + assert_eq!(rex.resident_composed_captures(display.mode()), expected); +} + fn assert_pixels_eq(label: &str, actual: &[u32], expected: &[u32]) { assert_eq!(actual.len(), expected.len(), "{label} length"); if let Some((index, (&actual, &expected))) = actual.iter().zip(expected).enumerate() @@ -1274,7 +1299,7 @@ fn resident_frame_gpu_marker_order_capture_and_display_changes() { gpu: None, accepts: false, outputs: Arc::clone(&outputs), resizes: Arc::clone(&resizes), })); - assert!(matches!(ResidentDisplay::new(&board.0, &gpu), + assert!(matches!(ResidentDisplay::new(&board.0, &gpu, ResidentDisplayMode::DirectWgpu), Err(ResidentError::Unsupported("resident frames require a wgpu presenter")))); let foreign = required_gpu(); assert!(!Arc::ptr_eq(&foreign, &gpu)); @@ -1282,13 +1307,16 @@ fn resident_frame_gpu_marker_order_capture_and_display_changes() { gpu: Some(foreign), accepts: true, outputs: Arc::clone(&outputs), resizes: Arc::clone(&resizes), })); - assert!(matches!(ResidentDisplay::new(&board.0, &gpu), Err(ResidentError::DeviceMismatch))); + assert!(matches!( + ResidentDisplay::new(&board.0, &gpu, ResidentDisplayMode::DirectWgpu), + Err(ResidentError::DeviceMismatch) + )); *board.0.head.renderer_slot().lock() = Some(Box::new(QueuePresenter { gpu: Some(Arc::clone(&gpu)), accepts: true, outputs: Arc::clone(&outputs), resizes: Arc::clone(&resizes), })); board.0.start(); - let mut display = ResidentDisplay::new(&board.0, &gpu).unwrap(); + let mut display = ResidentDisplay::new(&board.0, &gpu, ResidentDisplayMode::DirectWgpu).unwrap(); display.suppress_screenshot_files(); write_reg(&board.0, REX3_TOPSCAN, 1023, false); let mut first = fill_context(); @@ -1317,8 +1345,8 @@ fn resident_frame_gpu_marker_order_capture_and_display_changes() { assert_eq!(board.0.head.screen().lock().frame_seq, 0, "ordinary presentation did not capture"); assert_eq!(later_status.observations, ObservationCounters::default()); assert_eq!(later_status.raw_plane_map_attempts, 0); - assert_no_plane_traffic(display.traffic()); - assert_eq!(display.traffic().composed_readbacks.attempts, 0); + assert_no_plane_traffic(display.direct_traffic().unwrap()); + assert_eq!(display.capture_count().attempts, 0); assert_no_cpu_plane_access(&board.0); let expected_first = software_frame(&board.0, &first_rgb, &first_aux); @@ -1381,7 +1409,7 @@ fn resident_frame_gpu_marker_order_capture_and_display_changes() { assert_pixels_eq("requested composed frame", &captured, &expected_after_fence); board.0.request_screenshot(); - display.fail_next_capture(CaptureFault::AfterMap); + display.fail_next_capture(CaptureFault::AfterStaging); assert!(matches!(display.render(&board.0, std::time::Duration::from_secs(5), |_| {}), Err(ResidentError::Failed(_)))); assert!(board.0.screenshot_pending()); @@ -1396,7 +1424,7 @@ fn resident_frame_gpu_marker_order_capture_and_display_changes() { assert_pixels_eq("retried screenshot", &captured, &expected_after_fence); board.0.request_frame(); - display.fail_next_capture(CaptureFault::BeforeCopy); + display.fail_next_capture(CaptureFault::BeforeCapture); assert!(display.render(&board.0, std::time::Duration::from_secs(5), |_| {}).is_err()); assert!(board.0.frame_pending()); display.render(&board.0, std::time::Duration::from_secs(5), |_| {}).unwrap(); @@ -1405,7 +1433,7 @@ fn resident_frame_gpu_marker_order_capture_and_display_changes() { display.render(&board.0, std::time::Duration::from_secs(5), |_| {}).unwrap(); assert!(board.0.frame_available() && !board.0.frame_pending()); let continuous_seq = board.0.head.screen().lock().frame_seq; - display.fail_next_capture(CaptureFault::AfterMap); + display.fail_next_capture(CaptureFault::AfterStaging); assert!(display.render(&board.0, std::time::Duration::from_secs(5), |_| {}).is_err()); assert!(!board.0.frame_pending(), "continuous capture has no outstanding ticket"); assert!(!board.0.frame_available(), "failed continuous capture cannot expose stale pixels"); @@ -1440,7 +1468,7 @@ fn resident_frame_gpu_marker_order_capture_and_display_changes() { assert!(changed != expected_after_fence, "palette, gamma, cursor and topscan changes must be visible"); assert_eq!(crate::gfx_display::copy_frame(&*board.0, &mut captured).unwrap().0, WIDTH); assert_pixels_eq("changed display state", &captured, &changed); - let traffic = display.traffic(); + let traffic = display.direct_traffic().unwrap(); assert!(traffic.display.submitted > 4, "changed display generations must be uploaded"); assert_eq!(traffic.composed_readbacks.attempts, traffic.composed_readbacks.successes + 3); assert_no_plane_traffic(traffic); @@ -1563,7 +1591,8 @@ fn run_common_presenter_window_test() { assert!(board.0.head.renderer_slot().lock().as_deref().unwrap().takes_gpu_frames()); board.0.start(); - let mut display = ResidentDisplay::new(&board.0, &gpu).expect("bind common presenter"); + let mut display = ResidentDisplay::new(&board.0, &gpu, ResidentDisplayMode::DirectWgpu) + .expect("bind common presenter"); let mut fill = fill_context(); fill.drawmode0.0 |= 1 << 5; draw_reference(fill.clone(), &mut expected_rgb, &mut expected_aux); @@ -1618,8 +1647,8 @@ fn run_common_presenter_window_test() { ); assert_eq!(board.0.head.renderer_slot().lock().as_deref().unwrap().resident_presented_frames(), Some(3)); - assert_eq!(display.traffic().composed_readbacks.attempts, 0); - assert_no_plane_traffic(display.traffic()); + assert_eq!(display.capture_count().attempts, 0); + assert_no_plane_traffic(display.direct_traffic().unwrap()); let status = board.0.resident_status().unwrap(); assert_eq!(status.observations, ObservationCounters::default()); assert_eq!(status.raw_plane_map_attempts, 0); @@ -1672,29 +1701,152 @@ fn resident_frame_gpu_readback_compatibility_is_labeled() { let board = TestBoard(rex); let frames = Arc::new(Mutex::new(Vec::new())); let resizes = Arc::new(Mutex::new(Vec::new())); - *board.0.head.renderer_slot().lock() = Some(Box::new(CompatibilityPresenter { - frames: Arc::clone(&frames), - resizes: Arc::clone(&resizes), + let outputs = Arc::new(Mutex::new(Vec::new())); + let direct_resizes = Arc::new(Mutex::new(Vec::new())); + *board.0.head.renderer_slot().lock() = Some(Box::new(QueuePresenter { + gpu: Some(Arc::clone(&gpu)), + accepts: true, + outputs: Arc::clone(&outputs), + resizes: direct_resizes, })); board.0.start(); - let mut display = ResidentCompatibilityDisplay::new(&board.0) - .expect("create compatibility display"); + assert!(matches!( + ResidentDisplay::new(&board.0, &gpu, ResidentDisplayMode::ReadbackCompatibility), + Err(ResidentError::Unsupported("resident compatibility display requires a readback-capable renderer")) + )); + let mut direct = ResidentDisplay::new(&board.0, &gpu, ResidentDisplayMode::DirectWgpu).unwrap(); + direct.suppress_screenshot_files(); let mut fill = fill_context(); fill.drawmode0.0 |= 1 << 5; draw_reference(fill.clone(), &mut expected_rgb, &mut expected_aux); enqueue_fill(&board.0, &fill); + + let mut cpu = Display::new(); + cpu.screen.fb_rgb = expected_rgb.clone(); + cpu.screen.fb_aux = expected_aux.clone(); + let cpu_frames = Arc::new(Mutex::new(Vec::new())); + let mut cpu_presenter = CompatibilityPresenter { + frames: Arc::clone(&cpu_frames), + resizes: Arc::new(Mutex::new(Vec::new())), + }; + let stats = board.0.status.snapshot(Instant::now(), 0); + crate::dev::ng1::rex3::Renderer::present( + &mut cpu_presenter, + &mut cpu.screen, + &mut crate::debug_overlay::DebugOverlay::new(), + &mut crate::disp::StatusBar::new(), + &mut crate::disp::StatusBarTexture::new(), + &stats, + false, + None, + None, + ); + let expected = cpu_frames.lock()[0].clone(); + let mut cpu_scanout = Ng1Scanout::new(&gpu).expect("CPU-source wgpu scanout"); + let state = cpu.screen.display_state(); + let source = cpu_source(&state, &expected_rgb, &expected_aux); + let mut cpu_gpu = vec![0; WIDTH * HEIGHT]; + cpu_scanout.run(&source, &mut cpu_gpu).unwrap(); + let gpu_pixels: Vec = expected.iter().map(|pixel| pixel.swap_bytes() >> 8).collect(); + assert_pixels_eq("CPU source with wgpu scanout", &cpu_gpu, &gpu_pixels); + + direct.render(&board.0, std::time::Duration::from_secs(5), |_| {}) + .expect("resident direct presentation"); + let mut staging = Staging::default(); + let output = outputs.lock()[0].clone(); + let bytes = staging.read(&gpu, &output, &[(0, (WIDTH * HEIGHT * 4) as u64)]).unwrap(); + assert_pixels_eq("resident direct wgpu scanout", bytemuck::cast_slice(&bytes), &gpu_pixels); + assert_eq!(board.0.resident_status().unwrap().observations, ObservationCounters::default()); + assert_eq!(board.0.resident_status().unwrap().raw_plane_map_attempts, 0); + assert_eq!(direct.capture_count().attempts, 0); + + *board.0.head.renderer_slot().lock() = Some(Box::new(CompatibilityPresenter { + frames: Arc::clone(&frames), + resizes: Arc::clone(&resizes), + })); + board.0.request_frame(); + board.0.request_screenshot(); + board.0.set_capture(true); + assert!(matches!( + direct.render(&board.0, std::time::Duration::from_secs(5), |_| {}), + Err(ResidentError::Unsupported("resident frames require a wgpu presenter")) + )); + assert_eq!(direct.mode(), ResidentDisplayMode::DirectWgpu); + assert!(matches!( + board.0.start_resident_display(Arc::clone(&gpu), ResidentDisplayMode::DirectWgpu), + Err(ResidentError::Unsupported("resident frames require a wgpu presenter")) + )); + assert!(board.0.frame_pending() && board.0.screenshot_pending()); + assert_eq!(board.0.head.screen().lock().frame_seq, 0); + assert_eq!(board.0.resident_status().unwrap().observations, ObservationCounters::default()); + assert!(frames.lock().is_empty(), "direct rejection must not invoke compatibility"); + + *board.0.head.renderer_slot().lock() = None; + direct.render(&board.0, std::time::Duration::from_secs(5), |_| {}) + .expect("headless direct capture"); + assert_eq!(direct.mode(), ResidentDisplayMode::DirectWgpu); + assert_eq!(direct.capture_count(), super::direct::CaptureCount { + attempts: 1, successes: 1, + attempted_bytes: (WIDTH * HEIGHT * 4) as u64, + successful_bytes: (WIDTH * HEIGHT * 4) as u64, + }); + assert_composed_captures(&board.0, &direct, 1, 1); + assert_eq!(board.0.resident_composed_captures(ResidentDisplayMode::ReadbackCompatibility), Default::default()); + let mut captured = Vec::new(); + assert_eq!(crate::gfx_display::copy_frame(&*board.0, &mut captured).unwrap().2, 1); + assert_pixels_eq("headless direct capture", &captured, &expected); + assert!(!board.0.frame_pending() && !board.0.screenshot_pending()); + assert_eq!(board.0.resident_status().unwrap().observations, ObservationCounters::default()); + board.0.set_capture(false); + *board.0.head.renderer_slot().lock() = Some(Box::new(QueuePresenter { + gpu: Some(Arc::clone(&gpu)), + accepts: true, + outputs: Arc::clone(&outputs), + resizes: Arc::new(Mutex::new(Vec::new())), + })); + direct.render(&board.0, std::time::Duration::from_secs(5), |_| {}) + .expect("uncaptured direct presentation"); + let mut last_capture = Vec::new(); + assert_eq!(crate::gfx_display::copy_frame(&*board.0, &mut last_capture).unwrap().2, 1); + assert_pixels_eq("direct capture survives later presentation", &last_capture, &expected); + assert_composed_captures(&board.0, &direct, 1, 1); + drop(direct); + + *board.0.head.renderer_slot().lock() = Some(Box::new(CompatibilityPresenter { + frames: Arc::clone(&frames), + resizes: Arc::clone(&resizes), + })); + let mut display = ResidentDisplay::new(&board.0, &gpu, ResidentDisplayMode::ReadbackCompatibility) + .expect("explicitly select compatibility display"); + display.suppress_screenshot_files(); + board.0.request_frame(); + board.0.request_frame(); + board.0.request_screenshot(); + board.0.request_screenshot(); + board.0.set_capture(true); let position = display .render(&board.0, std::time::Duration::from_secs(5), |_| {}) .expect("render compatibility frame"); - - let expected = software_frame(&board.0, &expected_rgb, &expected_aux); - let frames = frames.lock(); - assert_eq!(frames.len(), 1); - assert_pixels_eq("readback compatibility frame", &frames[0], &expected); + assert_eq!(display.mode(), ResidentDisplayMode::ReadbackCompatibility); + assert!(display.direct_traffic().is_none()); + assert_eq!(display.capture_count(), super::direct::CaptureCount { + attempts: 1, successes: 1, + attempted_bytes: (WIDTH * HEIGHT * 4) as u64, + successful_bytes: (WIDTH * HEIGHT * 4) as u64, + }); + assert_composed_captures(&board.0, &display, 1, 1); + assert_eq!(board.0.resident_composed_captures(ResidentDisplayMode::DirectWgpu).successes, 1); + assert!(!board.0.frame_pending() && !board.0.screenshot_pending()); + assert_eq!(crate::gfx_display::copy_frame(&*board.0, &mut captured).unwrap().2, 2); + assert_pixels_eq("compatibility composed capture", &captured, &expected); + assert_eq!(frames.lock().len(), 1); + assert_pixels_eq("readback compatibility frame", &frames.lock()[0], &expected); assert_eq!(&*resizes.lock(), &[(WIDTH, HEIGHT)]); + assert_pixels_eq("CPU software presenter", &software_frame(&board.0, &expected_rgb, &expected_aux), &expected); + board.0.set_capture(false); let status = board.0.resident_status().unwrap(); - assert_eq!(position, status.submitted); + assert_eq!(position.position, status.submitted); let compatibility = status.observations.for_reason(ObservationReason::Compatibility); assert_eq!( ( @@ -1725,17 +1877,192 @@ fn resident_frame_gpu_readback_compatibility_is_labeled() { .unwrap_or_else(|panic| std::panic::resume_unwind(panic)); } +#[test] +#[ignore = "requires the requested conformant hardware GPU backend"] +fn resident_frame_gpu_compatibility_capture_failures_and_coalescing() { + std::thread::Builder::new() + .name("resident-compatibility-capture".into()) + .stack_size(64 * 1024 * 1024) + .spawn(|| { + use std::time::Duration; + + let gpu = required_gpu(); + let (rgb, aux) = seed_planes(); + let mut rex = Box::new(Rex3::new(crate::disp::StatusSource::detached())); + rex.fb_rgb.get_mut().copy_from_slice(&rgb); + rex.fb_aux.get_mut().copy_from_slice(&aux); + rex.fb_rgb.get_mut().isolate().unwrap(); + rex.fb_aux.get_mut().isolate().unwrap(); + configure_board_display(&rex); + rex.activate_resident_gpu(Arc::clone(&gpu)).unwrap(); + rex.fb_rgb.get_mut().reset_access_counts(); + rex.fb_aux.get_mut().reset_access_counts(); + let rgb_guard = unsafe { (&*rex.fb_rgb.get()).protect() }.unwrap(); + let aux_guard = unsafe { (&*rex.fb_aux.get()).protect() }.unwrap(); + let board = TestBoard(rex); + board.0.start(); + assert!(board.0.head.renderer_slot().lock().is_none()); + let mut display = ResidentDisplay::new(&board.0, &gpu, ResidentDisplayMode::ReadbackCompatibility).unwrap(); + display.suppress_screenshot_files(); + { + let mut screen = board.0.head.screen().lock(); + screen.rgba[0] = 0xff12_3456; + screen.frame_seq = 11; + } + board.0.request_frame(); + board.0.request_screenshot(); + board.0.set_capture(true); + display.fail_next_capture(CaptureFault::BeforeCapture); + assert!(matches!(display.render(&board.0, Duration::from_secs(5), |_| {}), Err(ResidentError::Failed(_)))); + assert!(board.0.frame_pending() && board.0.screenshot_pending()); + assert!(!board.0.frame_available()); + { + let screen = board.0.head.screen().lock(); + assert_eq!((screen.frame_seq, screen.rgba[0]), (11, 0xff12_3456)); + } + assert_compatibility_observations(&board.0, 1); + assert_composed_captures(&board.0, &display, 1, 0); + + display.render(&board.0, Duration::from_secs(5), |_| { + board.0.request_frame(); + board.0.request_screenshot(); + }).unwrap(); + assert!(board.0.frame_pending() && board.0.screenshot_pending()); + assert!(!board.0.frame_available(), "tickets newer than the cut remain pending"); + assert_eq!(board.0.head.screen().lock().frame_seq, 12); + assert_compatibility_observations(&board.0, 2); + assert_composed_captures(&board.0, &display, 2, 1); + display.render(&board.0, Duration::from_secs(5), |_| {}).unwrap(); + assert!(!board.0.frame_pending() && !board.0.screenshot_pending()); + let mut captured = Vec::new(); + assert_eq!(crate::gfx_display::copy_frame(&*board.0, &mut captured).unwrap().2, 13); + assert_pixels_eq("headless compatibility capture", &captured, &software_frame(&board.0, &rgb, &aux)); + assert_compatibility_observations(&board.0, 3); + assert_composed_captures(&board.0, &display, 3, 2); + + display.fail_next_capture(CaptureFault::AfterStaging); + assert!(matches!(display.render(&board.0, Duration::from_secs(5), |_| {}), Err(ResidentError::Failed(_)))); + assert!(!board.0.frame_pending() && !board.0.screenshot_pending()); + assert!(!board.0.frame_available(), "failed continuous capture cannot publish an old frame"); + { + let screen = board.0.head.screen().lock(); + assert_eq!(screen.frame_seq, 13); + for y in 0..HEIGHT { + assert_eq!(&screen.rgba[y * FB_STRIDE..y * FB_STRIDE + WIDTH], &captured[y * WIDTH..(y + 1) * WIDTH]); + } + } + assert_compatibility_observations(&board.0, 4); + assert_composed_captures(&board.0, &display, 4, 2); + display.render(&board.0, Duration::from_secs(5), |_| {}).unwrap(); + assert!(board.0.frame_available()); + assert_eq!(board.0.head.screen().lock().frame_seq, 14); + assert_compatibility_observations(&board.0, 5); + assert_composed_captures(&board.0, &display, 5, 3); + board.0.set_capture(false); + + board.0.request_frame(); + board.0.request_screenshot(); + let fence = board.0.gfifo_fence.load(Ordering::Acquire); + board.0.xmap_fence.store(fence.wrapping_add(1), Ordering::Release); + assert_eq!( + display.render(&board.0, Duration::ZERO, |_| {}), + Err(ResidentError::Timeout("display fence")) + ); + assert!(board.0.frame_pending() && board.0.screenshot_pending()); + assert_eq!(board.0.head.screen().lock().frame_seq, 14); + assert_compatibility_observations(&board.0, 5); + assert_composed_captures(&board.0, &display, 5, 3); + board.0.xmap_fence.store(fence, Ordering::Release); + display.render(&board.0, Duration::from_secs(5), |_| {}).unwrap(); + assert!(!board.0.frame_pending() && !board.0.screenshot_pending()); + assert_eq!(board.0.head.screen().lock().frame_seq, 15); + assert_compatibility_observations(&board.0, 6); + assert_composed_captures(&board.0, &display, 6, 4); + + board.0.request_frame(); + board.0.request_screenshot(); + let mode = (5 << DCBMODE_DCBADDR_SHIFT) + | ((crate::dev::ng1::xmap9::XMAP9_REG_MODE_TABLE_WRITE as u32) << DCBMODE_DCBCRS_SHIFT) + | DCBMODE_DATAWIDTH_4; + assert_eq!( + display.render(&board.0, Duration::from_secs(5), |_| { + write_reg(&board.0, REX3_DCBMODE, mode, false); + write_reg(&board.0, REX3_DCBDATA0, 0, false); + }), + Err(ResidentError::Timeout("display fence after frame marker")) + ); + assert!(board.0.frame_pending() && board.0.screenshot_pending()); + assert_eq!(board.0.head.screen().lock().frame_seq, 15); + assert_compatibility_observations(&board.0, 6); + assert_composed_captures(&board.0, &display, 6, 4); + board.0.wait_resident_display_fence(Duration::from_secs(5)).unwrap(); + display.render(&board.0, Duration::from_secs(5), |_| {}).unwrap(); + assert_eq!(board.0.head.screen().lock().frame_seq, 16); + assert_compatibility_observations(&board.0, 7); + assert_composed_captures(&board.0, &display, 7, 5); + + *board.0.head.renderer_slot().lock() = Some(Box::new(QueuePresenter { + gpu: Some(Arc::clone(&gpu)), accepts: true, + outputs: Arc::new(Mutex::new(Vec::new())), + resizes: Arc::new(Mutex::new(Vec::new())), + })); + board.0.request_frame(); + board.0.request_screenshot(); + assert_eq!( + display.render(&board.0, Duration::from_secs(5), |_| {}), + Err(ResidentError::Unsupported("resident compatibility display requires a readback-capable renderer")) + ); + assert_eq!(display.mode(), ResidentDisplayMode::ReadbackCompatibility); + assert!(board.0.frame_pending() && board.0.screenshot_pending()); + assert_eq!(board.0.head.screen().lock().frame_seq, 16); + assert_compatibility_observations(&board.0, 7); + assert_composed_captures(&board.0, &display, 7, 5); + *board.0.head.renderer_slot().lock() = None; + display.render(&board.0, Duration::from_secs(5), |_| {}).unwrap(); + assert!(!board.0.frame_pending() && !board.0.screenshot_pending()); + assert_eq!(board.0.head.screen().lock().frame_seq, 17); + assert_compatibility_observations(&board.0, 8); + assert_composed_captures(&board.0, &display, 8, 6); + let mut last_capture = Vec::new(); + assert_eq!(crate::gfx_display::copy_frame(&*board.0, &mut last_capture).unwrap().2, 17); + *board.0.head.renderer_slot().lock() = Some(Box::new(CompatibilityPresenter { + frames: Arc::new(Mutex::new(Vec::new())), + resizes: Arc::new(Mutex::new(Vec::new())), + })); + display.render(&board.0, Duration::from_secs(5), |_| {}).unwrap(); + let mut after_present = Vec::new(); + assert_eq!(crate::gfx_display::copy_frame(&*board.0, &mut after_present).unwrap().2, 17); + assert_pixels_eq("compatibility capture survives later presentation", &after_present, &last_capture); + assert_compatibility_observations(&board.0, 9); + assert_composed_captures(&board.0, &display, 8, 6); + assert_no_cpu_plane_access(&board.0); + board.0.stop(); + drop(aux_guard); + drop(rgb_guard); + }) + .expect("spawn compatibility capture test") + .join() + .unwrap_or_else(|panic| std::panic::resume_unwind(panic)); +} + #[test] #[ignore = "requires the requested conformant hardware GPU backend"] fn resident_frame_gpu_headless_screenshot_png_matches_cpu() { - const CHILD: &str = "IRIS_RESIDENT_SCREENSHOT_CHILD"; - if std::env::var_os(CHILD).is_some() { - std::thread::Builder::new().name("resident-screenshot".into()).stack_size(64 * 1024 * 1024) - .spawn(|| { - use std::time::Duration; - - let gpu = required_gpu(); - let (rgb, aux) = seed_planes(); + struct ScreenshotFile(std::path::PathBuf); + impl Drop for ScreenshotFile { + fn drop(&mut self) { + let _ = std::fs::remove_file(&self.0); + } + } + + std::thread::Builder::new().name("resident-screenshot".into()).stack_size(64 * 1024 * 1024) + .spawn(|| { + use std::io::Write; + use std::time::Duration; + + let gpu = required_gpu(); + let (rgb, aux) = seed_planes(); + for mode in [ResidentDisplayMode::DirectWgpu, ResidentDisplayMode::ReadbackCompatibility] { let mut rex = Box::new(Rex3::new(crate::disp::StatusSource::detached())); rex.fb_rgb.get_mut().copy_from_slice(&rgb); rex.fb_aux.get_mut().copy_from_slice(&aux); @@ -1750,14 +2077,31 @@ fn resident_frame_gpu_headless_screenshot_png_matches_cpu() { let board = TestBoard(rex); board.0.start(); assert!(board.0.head.renderer_slot().lock().is_none()); - let mut display = ResidentDisplay::new(&board.0, &gpu).unwrap(); + let mut display = ResidentDisplay::new(&board.0, &gpu, mode).unwrap(); + let mut number = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH) + .unwrap().as_nanos() as u32 ^ std::process::id(); + let path = loop { + let path = std::path::PathBuf::from(format!("screenshot_{number:04}.png")); + match std::fs::OpenOptions::new().write(true).create_new(true).open(&path) { + Ok(mut file) => { + let path = ScreenshotFile(path); + file.write_all(b"pending screenshot").unwrap(); + break path; + } + Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => { + number = number.wrapping_add(1); + } + Err(error) => panic!("reserve screenshot output: {error}"), + } + }; + board.0.screenshot_counter.store(number, Ordering::Relaxed); let initial_seq = board.0.head.screen().lock().frame_seq; board.0.request_screenshot(); - display.fail_next_capture(CaptureFault::BeforeCopy); + display.fail_next_capture(CaptureFault::BeforeCapture); assert!(display.render(&board.0, Duration::from_secs(10), |_| {}).is_err()); assert!(board.0.screenshot_pending()); assert_eq!(board.0.head.screen().lock().frame_seq, initial_seq); - assert!(!std::path::Path::new("screenshot_0000.png").exists()); + assert_eq!(std::fs::read(&path.0).unwrap(), b"pending screenshot"); display.render(&board.0, Duration::from_secs(10), |_| {}).unwrap(); assert!(!board.0.screenshot_pending()); assert_eq!(board.0.head.screen().lock().frame_seq, initial_seq + 1); @@ -1765,7 +2109,7 @@ fn resident_frame_gpu_headless_screenshot_png_matches_cpu() { let deadline = Instant::now() + Duration::from_secs(10); let (info, pixels) = loop { let decoded = (|| { - let file = std::fs::File::open("screenshot_0000.png").ok()?; + let file = std::fs::File::open(&path.0).ok()?; let mut reader = png::Decoder::new(std::io::BufReader::new(file)).read_info().ok()?; let mut pixels = vec![0; reader.output_buffer_size()?]; let info = reader.next_frame(&mut pixels).ok()?; @@ -1781,210 +2125,265 @@ fn resident_frame_gpu_headless_screenshot_png_matches_cpu() { .flat_map(|pixel| [*pixel as u8, (*pixel >> 8) as u8, (*pixel >> 16) as u8]) .collect(); assert_eq!(pixels, expected, "encoded screenshot differs from CPU composition"); - let traffic = display.traffic(); - assert_eq!((traffic.composed_readbacks.attempts, traffic.composed_readbacks.successes), (2, 1)); - assert_no_plane_traffic(traffic); - let status = board.0.resident_status().unwrap(); - assert_eq!(status.observations, ObservationCounters::default()); - assert_eq!(status.raw_plane_map_attempts, 0); + assert_composed_captures(&board.0, &display, 2, 1); + match mode { + ResidentDisplayMode::DirectWgpu => { + assert_no_plane_traffic(display.direct_traffic().unwrap()); + let status = board.0.resident_status().unwrap(); + assert_eq!(status.observations, ObservationCounters::default()); + assert_eq!(status.raw_plane_map_attempts, 0); + } + ResidentDisplayMode::ReadbackCompatibility => assert_compatibility_observations(&board.0, 2), + } assert_no_cpu_plane_access(&board.0); board.0.stop(); drop(aux_guard); drop(rgb_guard); - }).unwrap().join().unwrap_or_else(|panic| std::panic::resume_unwind(panic)); - return; - } - - let name = std::thread::current().name().unwrap().to_owned(); - let nonce = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos(); - let directory = std::env::current_dir().unwrap().join("target") - .join(format!("resident-screenshot-{}-{nonce}", std::process::id())); - std::fs::create_dir_all(directory.parent().unwrap()).unwrap(); - std::fs::create_dir(&directory).unwrap(); - let mut child = std::process::Command::new(std::env::current_exe().unwrap()) - .args(["--exact", &name, "--ignored", "--nocapture", "--test-threads=1"]) - .env(CHILD, "1") - .current_dir(&directory) - .stdout(std::process::Stdio::piped()) - .stderr(std::process::Stdio::piped()) - .spawn().expect("spawn resident screenshot test"); - let deadline = Instant::now() + std::time::Duration::from_secs(60); - while child.try_wait().unwrap().is_none() { - if Instant::now() >= deadline { - child.kill().unwrap(); - let output = child.wait_with_output().unwrap(); - std::fs::remove_dir_all(&directory).unwrap(); - panic!("resident screenshot test timed out: {}", String::from_utf8_lossy(&output.stderr)); - } - std::thread::sleep(std::time::Duration::from_millis(10)); - } - let output = child.wait_with_output().unwrap(); - std::fs::remove_dir_all(&directory).unwrap(); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!(output.status.success() && stdout.contains("1 passed; 0 failed"), "{stdout}\n{stderr}"); + } + }).unwrap().join().unwrap_or_else(|panic| std::panic::resume_unwind(panic)); } #[test] #[ignore = "requires the requested conformant hardware GPU backend"] fn resident_frame_gpu_headless_requests_and_vnc_capture() { - std::thread::Builder::new().name("resident-frame-headless".into()).stack_size(64 * 1024 * 1024) + std::thread::Builder::new() + .name("resident-frame-headless".into()) + .stack_size(64 * 1024 * 1024) .spawn(|| { use std::io::{Read, Write}; use std::time::{Duration, Instant}; let gpu = required_gpu(); let (rgb, aux) = seed_planes(); - let mut rex = Box::new(Rex3::new(crate::disp::StatusSource::detached())); - rex.fb_rgb.get_mut().copy_from_slice(&rgb); - rex.fb_aux.get_mut().copy_from_slice(&aux); - rex.fb_rgb.get_mut().isolate().unwrap(); - rex.fb_aux.get_mut().isolate().unwrap(); - configure_board_display(&rex); - rex.activate_resident_gpu(Arc::clone(&gpu)).unwrap(); - unsafe { - (*rex.fb_rgb.get()).reset_access_counts(); - (*rex.fb_aux.get()).reset_access_counts(); - } - let rgb_guard = unsafe { (&*rex.fb_rgb.get()).protect() }.unwrap(); - let aux_guard = unsafe { (&*rex.fb_aux.get()).protect() }.unwrap(); - let board = SharedTestBoard(Arc::from(rex)); - board.0.start(); - write_reg(&board.0, REX3_TOPSCAN, 1023, false); - assert!(board.0.head.renderer_slot().lock().is_none()); - let mut headless = ResidentDisplay::new(&board.0, &gpu).unwrap(); - headless.suppress_screenshot_files(); - board.0.request_screenshot(); - headless.render(&board.0, Duration::from_secs(10), |_| {}).unwrap(); - assert!(!board.0.screenshot_pending()); - let mut screenshot = Vec::new(); - assert!(crate::gfx_display::copy_frame(&*board.0, &mut screenshot).is_some()); - assert_pixels_eq("headless screenshot", &screenshot, &software_frame(&board.0, &rgb, &aux)); - - let mut oracle = Display::new(); - oracle.screen.topscan = 1023; - for (did, line) in [(6, DID_LINE as u16), (2, 0xffff)] { - for vc2 in [&board.0.vc2, &oracle.vc2] { - let mut vc2 = vc2.lock(); - vc2.ram[DID_LINE] = did; - vc2.ram[DID_LINE + 1] = 0xffe0; - vc2.ram[0x300 + 3] = line; - vc2.dirty = true; + for mode in [ + ResidentDisplayMode::DirectWgpu, + ResidentDisplayMode::ReadbackCompatibility, + ] { + let mut rex = Box::new(Rex3::new(crate::disp::StatusSource::detached())); + rex.fb_rgb.get_mut().copy_from_slice(&rgb); + rex.fb_aux.get_mut().copy_from_slice(&aux); + rex.fb_rgb.get_mut().isolate().unwrap(); + rex.fb_aux.get_mut().isolate().unwrap(); + configure_board_display(&rex); + rex.activate_resident_gpu(Arc::clone(&gpu)).unwrap(); + unsafe { + (*rex.fb_rgb.get()).reset_access_counts(); + (*rex.fb_aux.get()).reset_access_counts(); } - oracle.refresh(); - board.0.request_frame(); - headless.render(&board.0, Duration::from_secs(10), |_| {}).unwrap(); + let rgb_guard = unsafe { (&*rex.fb_rgb.get()).protect() }.unwrap(); + let aux_guard = unsafe { (&*rex.fb_aux.get()).protect() }.unwrap(); + let board = SharedTestBoard(Arc::from(rex)); + board.0.start(); + write_reg(&board.0, REX3_TOPSCAN, 1023, false); + assert!(board.0.head.renderer_slot().lock().is_none()); + let mut headless = ResidentDisplay::new(&board.0, &gpu, mode).unwrap(); + headless.suppress_screenshot_files(); + board.0.request_screenshot(); + headless + .render(&board.0, Duration::from_secs(10), |_| {}) + .unwrap(); + assert!(!board.0.screenshot_pending()); + let mut screenshot = Vec::new(); assert!(crate::gfx_display::copy_frame(&*board.0, &mut screenshot).is_some()); assert_pixels_eq( - "headless DID history", + "headless screenshot", &screenshot, + &software_frame(&board.0, &rgb, &aux), + ); + + let mut oracle = Display::new(); + oracle.screen.topscan = 1023; + for (did, line) in [(6, DID_LINE as u16), (2, 0xffff)] { + for vc2 in [&board.0.vc2, &oracle.vc2] { + let mut vc2 = vc2.lock(); + vc2.ram[DID_LINE] = did; + vc2.ram[DID_LINE + 1] = 0xffe0; + vc2.ram[0x300 + 3] = line; + vc2.dirty = true; + } + oracle.refresh(); + board.0.request_frame(); + headless + .render(&board.0, Duration::from_secs(10), |_| {}) + .unwrap(); + assert!(crate::gfx_display::copy_frame( + &*board.0, + &mut screenshot + ) + .is_some()); + assert_pixels_eq( + "headless DID history", + &screenshot, + &software_display_frame(&mut oracle, &rgb, &aux), + ); + } + assert_eq!(oracle.screen.did[3 * FB_STRIDE], 6); + if mode == ResidentDisplayMode::DirectWgpu { + assert!(board.0.head.screen().lock().did.iter().all(|&did| did == 0)); + } + let expected = software_display_frame(&mut oracle, &rgb, &aux); + drop(headless); + board + .0 + .start_resident_display(Arc::clone(&gpu), mode) + .unwrap(); + + let mut captured = Vec::new(); + let (width, height, seq) = crate::gfx_display::fresh_frame( + &*board.0, + &mut captured, + Duration::from_secs(10), + ) + .expect("headless composed frame"); + assert_eq!((width, height), (WIDTH, HEIGHT)); + assert_pixels_eq("headless composed frame", &captured, &expected); + let initial = captured.clone(); + let cursor_y = { + let mut vc2 = board.0.vc2.lock(); + vc2.regs[VC2_REG_CURSOR_Y_LOC as usize] += 7; + vc2.dirty = true; + vc2.regs[VC2_REG_CURSOR_Y_LOC as usize] + }; + let deadline = Instant::now() + Duration::from_secs(5); + while board.0.vc2.lock().regs[VC2_REG_WORKING_CURSOR_Y as usize] != cursor_y + && Instant::now() < deadline + { + std::thread::sleep(Duration::from_millis(2)); + } + assert_eq!( + board.0.vc2.lock().regs[VC2_REG_WORKING_CURSOR_Y as usize], + cursor_y + ); + let (_, _, updated_seq) = crate::gfx_display::fresh_frame( + &*board.0, + &mut captured, + Duration::from_secs(10), + ) + .expect("headless cursor refresh"); + assert!(updated_seq > seq); + assert!( + captured != initial, + "vblank cursor changes must reach the composed frame" + ); + { + let mut vc2 = oracle.vc2.lock(); + vc2.regs[VC2_REG_WORKING_CURSOR_Y as usize] = cursor_y; + vc2.dirty = true; + } + oracle.refresh(); + assert_pixels_eq( + "headless cursor update", + &captured, &software_display_frame(&mut oracle, &rgb, &aux), ); - } - assert_eq!(oracle.screen.did[3 * FB_STRIDE], 6); - assert!(board.0.head.screen().lock().did.iter().all(|&did| did == 0)); - let expected = software_display_frame(&mut oracle, &rgb, &aux); - drop(headless); - board.0.start_resident_display(Arc::clone(&gpu)).unwrap(); + let seq = updated_seq; + std::thread::sleep(Duration::from_millis(100)); + assert_eq!( + board.0.head.screen().lock().frame_seq, + seq, + "headless idle must not perform an unrequested readback" + ); - let mut captured = Vec::new(); - let (width, height, seq) = crate::gfx_display::fresh_frame( - &*board.0, &mut captured, Duration::from_secs(10), - ).expect("headless composed frame"); - assert_eq!((width, height), (WIDTH, HEIGHT)); - assert_pixels_eq("headless composed frame", &captured, &expected); - let initial = captured.clone(); - let cursor_y = { - let mut vc2 = board.0.vc2.lock(); - vc2.regs[VC2_REG_CURSOR_Y_LOC as usize] += 7; - vc2.dirty = true; - vc2.regs[VC2_REG_CURSOR_Y_LOC as usize] - }; - let deadline = Instant::now() + Duration::from_secs(5); - while board.0.vc2.lock().regs[VC2_REG_WORKING_CURSOR_Y as usize] != cursor_y - && Instant::now() < deadline - { - std::thread::sleep(Duration::from_millis(2)); - } - assert_eq!(board.0.vc2.lock().regs[VC2_REG_WORKING_CURSOR_Y as usize], cursor_y); - let (_, _, updated_seq) = crate::gfx_display::fresh_frame( - &*board.0, &mut captured, Duration::from_secs(10), - ).expect("headless cursor refresh"); - assert!(updated_seq > seq); - assert!(captured != initial, "vblank cursor changes must reach the composed frame"); - { - let mut vc2 = oracle.vc2.lock(); - vc2.regs[VC2_REG_WORKING_CURSOR_Y as usize] = cursor_y; - vc2.dirty = true; - } - oracle.refresh(); - assert_pixels_eq("headless cursor update", &captured, &software_display_frame(&mut oracle, &rgb, &aux)); - let seq = updated_seq; - std::thread::sleep(Duration::from_millis(100)); - assert_eq!(board.0.head.screen().lock().frame_seq, seq, - "headless idle must not perform an unrequested readback"); - - let display: Arc = board.0.clone(); - let ps2 = Arc::new(crate::dev::ps2::Ps2Controller::new(None)); - let input = Arc::new(crate::hostinput::HostInput::new(Some(display), ps2)); - let server = crate::vnc::VncServer::start(input, "127.0.0.1:0", Some(30), "resident").unwrap(); - let mut stream = std::net::TcpStream::connect(server.addr()).unwrap(); - stream.set_read_timeout(Some(Duration::from_secs(10))).unwrap(); - let mut version = [0; 12]; - stream.read_exact(&mut version).unwrap(); - assert_eq!(&version, b"RFB 003.008\n"); - stream.write_all(&version).unwrap(); - let mut security = [0; 2]; - stream.read_exact(&mut security).unwrap(); - assert_eq!(security, [1, 1]); - stream.write_all(&[1]).unwrap(); - let mut security_result = [0; 4]; - stream.read_exact(&mut security_result).unwrap(); - assert_eq!(security_result, [0; 4]); - stream.write_all(&[1]).unwrap(); - let mut init = [0; 24]; - stream.read_exact(&mut init).unwrap(); - assert_eq!(u16::from_be_bytes([init[0], init[1]]) as usize, WIDTH); - assert_eq!(u16::from_be_bytes([init[2], init[3]]) as usize, HEIGHT); - let name_len = u32::from_be_bytes(init[20..24].try_into().unwrap()) as usize; - let mut name = vec![0; name_len]; - stream.read_exact(&mut name).unwrap(); - assert_eq!(&name, b"resident"); - - let mut request = vec![3, 0, 0, 0, 0, 0]; - request.extend_from_slice(&(WIDTH as u16).to_be_bytes()); - request.extend_from_slice(&(HEIGHT as u16).to_be_bytes()); - stream.write_all(&request).unwrap(); - let mut update = [0; 4]; - stream.read_exact(&mut update).unwrap(); - assert_eq!(update, [0, 0, 0, 1]); - let mut rect = [0; 12]; - stream.read_exact(&mut rect).unwrap(); - assert_eq!(&rect[..4], &[0; 4]); - assert_eq!(u16::from_be_bytes([rect[4], rect[5]]) as usize, WIDTH); - assert_eq!(u16::from_be_bytes([rect[6], rect[7]]) as usize, HEIGHT); - assert_eq!(i32::from_be_bytes(rect[8..12].try_into().unwrap()), 0); - let mut pixels = vec![0; WIDTH * HEIGHT * 4]; - stream.read_exact(&mut pixels).unwrap(); - let vnc: Vec = pixels.chunks_exact(4) - .map(|pixel| u32::from_le_bytes(pixel.try_into().unwrap())).collect(); - let expected: Vec = captured.iter().map(|pixel| pixel & 0x00ff_ffff).collect(); - assert_pixels_eq("VNC composed frame", &vnc, &expected); - assert!(board.0.head.screen().lock().frame_seq > seq, "VNC enabled composed capture"); - drop(stream); - let deadline = Instant::now() + Duration::from_secs(5); - while server.clients() != 0 && Instant::now() < deadline { - std::thread::sleep(Duration::from_millis(10)); + let display: Arc = board.0.clone(); + let ps2 = Arc::new(crate::dev::ps2::Ps2Controller::new(None)); + let input = Arc::new(crate::hostinput::HostInput::new(Some(display), ps2)); + let server = + crate::vnc::VncServer::start(input, "127.0.0.1:0", Some(30), "resident") + .unwrap(); + let mut stream = std::net::TcpStream::connect(server.addr()).unwrap(); + stream + .set_read_timeout(Some(Duration::from_secs(10))) + .unwrap(); + let mut version = [0; 12]; + stream.read_exact(&mut version).unwrap(); + assert_eq!(&version, b"RFB 003.008\n"); + stream.write_all(&version).unwrap(); + let mut security = [0; 2]; + stream.read_exact(&mut security).unwrap(); + assert_eq!(security, [1, 1]); + stream.write_all(&[1]).unwrap(); + let mut security_result = [0; 4]; + stream.read_exact(&mut security_result).unwrap(); + assert_eq!(security_result, [0; 4]); + stream.write_all(&[1]).unwrap(); + let mut init = [0; 24]; + stream.read_exact(&mut init).unwrap(); + assert_eq!(u16::from_be_bytes([init[0], init[1]]) as usize, WIDTH); + assert_eq!(u16::from_be_bytes([init[2], init[3]]) as usize, HEIGHT); + let name_len = u32::from_be_bytes(init[20..24].try_into().unwrap()) as usize; + let mut name = vec![0; name_len]; + stream.read_exact(&mut name).unwrap(); + assert_eq!(&name, b"resident"); + + let mut request = vec![3, 0, 0, 0, 0, 0]; + request.extend_from_slice(&(WIDTH as u16).to_be_bytes()); + request.extend_from_slice(&(HEIGHT as u16).to_be_bytes()); + stream.write_all(&request).unwrap(); + let mut update = [0; 4]; + stream.read_exact(&mut update).unwrap(); + assert_eq!(update, [0, 0, 0, 1]); + let mut rect = [0; 12]; + stream.read_exact(&mut rect).unwrap(); + assert_eq!(&rect[..4], &[0; 4]); + assert_eq!(u16::from_be_bytes([rect[4], rect[5]]) as usize, WIDTH); + assert_eq!(u16::from_be_bytes([rect[6], rect[7]]) as usize, HEIGHT); + assert_eq!(i32::from_be_bytes(rect[8..12].try_into().unwrap()), 0); + let mut pixels = vec![0; WIDTH * HEIGHT * 4]; + stream.read_exact(&mut pixels).unwrap(); + let vnc: Vec = pixels + .chunks_exact(4) + .map(|pixel| u32::from_le_bytes(pixel.try_into().unwrap())) + .collect(); + let expected: Vec = captured.iter().map(|pixel| pixel & 0x00ff_ffff).collect(); + assert_pixels_eq("VNC composed frame", &vnc, &expected); + assert!( + board.0.head.screen().lock().frame_seq > seq, + "VNC enabled composed capture" + ); + drop(stream); + let deadline = Instant::now() + Duration::from_secs(5); + while server.clients() != 0 && Instant::now() < deadline { + std::thread::sleep(Duration::from_millis(10)); + } + assert_eq!(server.clients(), 0, "VNC client must disconnect"); + drop(server); + board.0.stop(); + let frames = board.0.head.screen().lock().frame_seq; + let bytes = (WIDTH * HEIGHT * 4) as u64; + assert_eq!( + board.0.resident_composed_captures(mode), + super::direct::CaptureCount { + attempts: frames, + successes: frames, + attempted_bytes: frames * bytes, + successful_bytes: frames * bytes, + }, + "worker-owned captures must remain observable after shutdown" + ); + let status = board.0.resident_status().unwrap(); + match mode { + ResidentDisplayMode::DirectWgpu => { + assert_eq!(status.observations, ObservationCounters::default()); + assert_eq!(status.raw_plane_map_attempts, 0); + } + ResidentDisplayMode::ReadbackCompatibility => { + let frames = status + .observations + .for_reason(ObservationReason::Compatibility) + .attempts; + assert!( + frames >= 4, + "manual capture, CI frames and VNC must all observe" + ); + assert_compatibility_observations(&board.0, frames); + } + } + assert_no_cpu_plane_access(&board.0); + drop(aux_guard); + drop(rgb_guard); } - assert_eq!(server.clients(), 0, "VNC client must disconnect"); - drop(server); - board.0.stop(); - let status = board.0.resident_status().unwrap(); - assert_eq!(status.observations, ObservationCounters::default()); - assert_eq!(status.raw_plane_map_attempts, 0); - assert_no_cpu_plane_access(&board.0); - drop(aux_guard); - drop(rgb_guard); - }).expect("spawn resident headless integration") - .join().unwrap_or_else(|panic| std::panic::resume_unwind(panic)); + }) + .expect("spawn resident headless integration") + .join() + .unwrap_or_else(|panic| std::panic::resume_unwind(panic)); } diff --git a/src/dev/ng1/wgpu/resident.rs b/src/dev/ng1/wgpu/resident.rs index b1f12a946..aa2b558db 100644 --- a/src/dev/ng1/wgpu/resident.rs +++ b/src/dev/ng1/wgpu/resident.rs @@ -14,12 +14,16 @@ use parking_lot::Mutex; use crate::dev::ng1::rex3::{ Rex3Context, DRAWMODE0_ADRMODE_A_LINE, DRAWMODE0_ADRMODE_F_LINE, DRAWMODE0_ADRMODE_I_LINE, DRAWMODE0_OPCODE_DRAW, DRAWMODE0_OPCODE_NOOP, - DRAWMODE0_OPCODE_SCR2SCR, DRAWMODE1_LOGICOP_SRC, OCTANT_XDEC, OCTANT_YDEC, + DRAWMODE0_OPCODE_READ, DRAWMODE0_OPCODE_SCR2SCR, DRAWMODE1_LOGICOP_SRC, + DRAWMODE1_PLANES_CID, DRAWMODE1_PLANES_OLAY, DRAWMODE1_PLANES_PUP, + DRAWMODE1_PLANES_RGB, DRAWMODE1_PLANES_RGBA, OCTANT_XDEC, OCTANT_YDEC, REX3_SCREEN_HEIGHT, REX3_SCREEN_WIDTH, }; use crate::dev::ng1::rex3_flat_fill::{ prepare_resident_flat_fill, FillKind, FillPlane, }; +use crate::dev::ng1::rex3_generic::{draw_with_fb, Framebuffers}; +use crate::dev::ng1::rex3_shape; use crate::wgpu::host::HostBuffer; use crate::wgpu::tiles::{COARSE, LIST_CAP, MASK_WORDS, TILE_H, TILE_W}; use crate::wgpu::{ArchShaders, Batch, BatchPipelines, Fragment, Full, Gpu, TileGrid}; @@ -593,6 +597,13 @@ pub(crate) trait ResidentBackend: Send { pub(crate) struct ResidentRaster { backend: Box, fault: Option, + host_read: HostReadSnapshot, +} + +#[derive(Default)] +struct HostReadSnapshot { + key: Option<(FillPlane, Position)>, + words: Vec, } #[derive(Clone, Debug)] @@ -609,7 +620,11 @@ pub(crate) enum ResidentFaultInjection { impl ResidentRaster { pub(crate) fn new(backend: impl ResidentBackend + 'static) -> Self { - Self { backend: Box::new(backend), fault: None } + Self { + backend: Box::new(backend), + fault: None, + host_read: HostReadSnapshot::default(), + } } pub(crate) fn gpu(&self) -> Option> { @@ -626,7 +641,7 @@ impl ResidentRaster { }) } - pub(crate) fn enqueue(&mut self, context: &mut Rex3Context) -> Result<(), ResidentError> { + fn enqueue_backend(&mut self, context: &mut Rex3Context) -> Result<(), ResidentError> { if let Some(error) = &self.fault { return Err(error.clone()); } @@ -646,6 +661,100 @@ impl ResidentRaster { } } + fn host_read_plane(context: &Rex3Context) -> Option { + match context.drawmode1.planes() { + DRAWMODE1_PLANES_RGB | DRAWMODE1_PLANES_RGBA => Some(FillPlane::Rgb), + DRAWMODE1_PLANES_OLAY | DRAWMODE1_PLANES_PUP | DRAWMODE1_PLANES_CID => { + Some(FillPlane::Aux) + } + _ => None, + } + } + + fn prepare_host_read_snapshot(&mut self, plane: FillPlane) -> Result<(), ResidentError> { + let status = self.status()?; + let generation = match plane { + FillPlane::Rgb => status.rgb.last_write, + FillPlane::Aux => status.aux.last_write, + }; + if self.host_read.key != Some((plane, generation)) { + let mask = match plane { + FillPlane::Rgb => PlaneMask::Rgb, + FillPlane::Aux => PlaneMask::Aux, + }; + let observation = self.observe(ObservationReason::HostRead, mask)?; + let (bytes, observed_generation) = match plane { + FillPlane::Rgb => (observation.rgb(), observation.rgb_generation()), + FillPlane::Aux => (observation.aux(), observation.aux_generation()), + }; + if observed_generation != Some(generation) { + return Err(ResidentError::InvalidState( + "HostR snapshot generation changed", + )); + } + let bytes = bytes.ok_or(ResidentError::InvalidState( + "HostR snapshot omitted its selected plane", + ))?; + if bytes.len() != PLANE_BYTES as usize { + return Err(ResidentError::InvalidState( + "HostR snapshot has the wrong size", + )); + } + self.host_read.words.resize(PLANE_WORDS, 0); + for (word, bytes) in self + .host_read + .words + .iter_mut() + .zip(bytes.chunks_exact(size_of::())) + { + *word = u32::from_ne_bytes(bytes.try_into().unwrap()); + } + self.host_read.key = Some((plane, generation)); + } + Ok(()) + } + + fn enqueue_host_read(&mut self, context: &mut Rex3Context) -> Result<(), ResidentError> { + if let Some(plane) = Self::host_read_plane(context) { + self.prepare_host_read_snapshot(plane)?; + } else { + self.host_read.words.resize(PLANE_WORDS, 0); + } + self.enqueue_backend(context)?; + let mode = rex3_shape::unpack( + context.drawmode0.0, + context.drawmode1.0, + context.clipmode, + ); + { + let pointer = self.host_read.words.as_mut_ptr(); + let framebuffers = Framebuffers { + rgb: pointer, + aux: pointer, + }; + draw_with_fb(context, &framebuffers, &mode); + while context.host_len > 1 && context.host_cursor < context.host_len { + let before = context.host_cursor; + context.hostcnt = 0; + draw_with_fb(context, &framebuffers, &mode); + if context.host_cursor == before { + break; + } + } + } + context.host_len = 1; + context.host_cursor = 0; + Ok(()) + } + + pub(crate) fn enqueue(&mut self, context: &mut Rex3Context) -> Result<(), ResidentError> { + if context.drawmode0.opcode() == DRAWMODE0_OPCODE_READ { + self.enqueue_host_read(context) + } else { + self.enqueue_backend(context) + } + } + pub(crate) fn submit(&mut self) -> Result<(), ResidentError> { self.run(|backend| backend.submit()) } @@ -1456,6 +1565,11 @@ impl ResidentBackend for ResidentGpu { context.host_cursor = 0; return Ok(()); } + if opcode == DRAWMODE0_OPCODE_READ { + let position = self.progress.lock().next_position()?; + self.progress.lock().accept(position, None); + return Ok(()); + } if opcode == DRAWMODE0_OPCODE_SCR2SCR { return self.enqueue_copy(context); } diff --git a/src/dev/ng1/wgpu/resident_tests.rs b/src/dev/ng1/wgpu/resident_tests.rs index 8cc45993a..66f2c9364 100644 --- a/src/dev/ng1/wgpu/resident_tests.rs +++ b/src/dev/ng1/wgpu/resident_tests.rs @@ -520,14 +520,59 @@ fn resident_gpu_observation_selection_positions_and_shadow_acknowledgement() { assert_eq!(aux.aux_generation(), both.aux_generation()); assert!(aux.rgb().is_none()); compare_observed(aux.aux().unwrap(), &expected_aux, "selected aux"); - let counts = raster.status().unwrap(); - assert_eq!(counts.raw_plane_map_attempts, 4); + let before_hostr = raster.status().unwrap(); + assert_eq!(before_hostr.raw_plane_map_attempts, 4); for reason in [ObservationReason::HostRead, ObservationReason::Monitor, ObservationReason::Validation] { - let count = counts.observations.for_reason(reason); + let count = before_hostr.observations.for_reason(reason); assert_eq!((count.attempts, count.successes), (1, 1)); assert_eq!((count.attempted_transfers, count.successful_transfers), (1, 1)); assert_eq!((count.attempted_bytes, count.successful_bytes), (PLANE_BYTES, PLANE_BYTES)); } + + let read_context = |plane, rgb_mode, start, end| { + let mut context = context( + plane, + rgb_mode, + true, + start, + end, + u32::MAX, + 0, + ); + context.drawmode0 = + DrawMode0(DRAWMODE0_OPCODE_READ | DRAWMODE0_ADRMODE_SPAN_SH | (1 << 8)); + context.drawmode1.0 |= (1 << 7) | (3 << 8) | (1 << 10); + context.host_len = 1; + context.host_cursor = 0; + context.hostcnt = 0; + context.host_shifter = 0; + context + }; + for (label, plane, rgb_mode, start, end) in [ + ("first RGB HostR", DRAWMODE1_PLANES_RGB, true, (3, 1), (4, 1)), + ("reused RGB HostR", DRAWMODE1_PLANES_RGB, true, (8, 2), (9, 2)), + ("first auxiliary HostR", DRAWMODE1_PLANES_CID, false, (4, 4), (5, 4)), + ("reused auxiliary HostR", DRAWMODE1_PLANES_CID, false, (4, 5), (5, 5)), + ] { + let mut actual = read_context(plane, rgb_mode, start, end); + let mut expected = actual; + raster.enqueue(&mut actual).expect(label); + cpu_host_draw(&mut expected, &mut expected_rgb, &mut expected_aux); + assert_context_eq(&actual, &expected, label); + } + let counts = raster.status().unwrap(); + assert_eq!(counts.accepted.sequence, Sequence(9)); + assert_eq!(counts.raw_plane_map_attempts, 6); + let hostr = counts.observations.for_reason(ObservationReason::HostRead); + assert_eq!((hostr.attempts, hostr.successes), (3, 3)); + assert_eq!( + (hostr.attempted_transfers, hostr.successful_transfers), + (3, 3) + ); + assert_eq!( + (hostr.attempted_bytes, hostr.successful_bytes), + (3 * PLANE_BYTES, 3 * PLANE_BYTES) + ); assert_eq!(resident.lock().readback, staging, "one staging allocation serves every selection"); shadow_rgb.copy_from_slice(&seed_rgb); @@ -638,26 +683,35 @@ fn resident_gpu_partial_restore_keeps_suffix_and_rebases_stream() { #[test] #[ignore = "requires the requested conformant host GPU"] fn resident_gpu_failed_observation_is_counted_and_latched() { - struct FailingObservation(Arc>); + struct FailingObservation { + resident: Arc>, + gpu: Arc, + } impl ResidentBackend for FailingObservation { + fn gpu(&self) -> Option<&Arc> { Some(&self.gpu) } + fn enqueue(&mut self, context: &mut Rex3Context) -> Result<(), ResidentError> { - self.0.lock().enqueue(context) + self.resident.lock().enqueue(context) } fn submit(&mut self) -> Result<(), ResidentError> { - self.0.lock().submit() + self.resident.lock().submit() } fn observe(&mut self, reason: ObservationReason, planes: PlaneMask) -> Result { - self.0.lock().observe_using(reason, planes, |backend, planes| { + self.resident.lock().observe_using(reason, planes, |backend, planes| { let _ = backend.read_planes(planes)?; Err(ResidentError::Failed("injected observation failure after map".into())) }) } + fn status(&mut self) -> Result { + self.resident.lock().status() + } + fn recorded_status(&self) -> Result { - self.0.lock().recorded_status() + self.resident.lock().recorded_status() } } @@ -666,7 +720,10 @@ fn resident_gpu_failed_observation_is_counted_and_latched() { let rgb = seeded_plane(0x0011_2233); let aux = seeded_plane(0xaabb_ccdd); let resident = Arc::new(Mutex::new(ResidentGpu::activate(Arc::clone(&gpu), &rgb, &aux).unwrap())); - let mut raster = ResidentRaster::new(FailingObservation(Arc::clone(&resident))); + let mut raster = ResidentRaster::new(FailingObservation { + resident: Arc::clone(&resident), + gpu: Arc::clone(&gpu), + }); let failure = ResidentError::Failed("injected observation failure after map".into()); assert_eq!(raster.observe(ObservationReason::Snapshot, PlaneMask::Both).unwrap_err(), failure); let diagnostics = raster.diagnostics().unwrap(); @@ -687,6 +744,66 @@ fn resident_gpu_failed_observation_is_counted_and_latched() { ObservationCounts::default(), "a request refused by the sticky fault is not a transfer attempt", ); + + drop(raster); + drop(resident); + let mut board = TestRex(Box::new(Rex3::new(crate::disp::StatusSource::detached()))); + board.0.fb_rgb.get_mut().copy_from_slice(&rgb); + board.0.fb_aux.get_mut().copy_from_slice(&aux); + board.0.fb_rgb.get_mut().isolate().unwrap(); + board.0.fb_aux.get_mut().isolate().unwrap(); + let resident = Arc::new(Mutex::new(ResidentGpu::activate( + Arc::clone(&gpu), board.0.fb_rgb.get_mut(), board.0.fb_aux.get_mut(), + ).unwrap())); + board.0.attach_resident_raster(FailingObservation { + resident: Arc::clone(&resident), + gpu: Arc::clone(&gpu), + }).unwrap(); + board.0.fb_rgb.get_mut().reset_access_counts(); + board.0.fb_aux.get_mut().reset_access_counts(); + let rgb_guard = unsafe { (&*board.0.fb_rgb.get()).protect() }.unwrap(); + let aux_guard = unsafe { (&*board.0.fb_aux.get()).protect() }.unwrap(); + { + let mut screen = board.0.head.screen().lock(); + screen.width = 2; + screen.height = 2; + screen.rgba[0] = 0xff12_3456; + screen.frame_seq = 7; + } + use crate::dev::ng1::rex3::frame::{ResidentDisplay, ResidentDisplayMode}; + use crate::gfx_display::GfxDisplay; + board.0.start(); + let mut display = ResidentDisplay::new(&board.0, &gpu, ResidentDisplayMode::ReadbackCompatibility).unwrap(); + display.suppress_screenshot_files(); + board.0.request_frame(); + board.0.request_screenshot(); + assert_eq!( + display.render(&board.0, std::time::Duration::from_secs(5), |_| {}), + Err(failure.clone()), + ); + assert!(board.0.frame_pending() && board.0.screenshot_pending()); + assert!(!board.0.frame_available()); + { + let screen = board.0.head.screen().lock(); + assert_eq!((screen.frame_seq, screen.rgba[0]), (7, 0xff12_3456)); + } + assert_eq!(display.capture_count(), Default::default()); + assert_eq!( + board.0.resident_composed_captures(ResidentDisplayMode::ReadbackCompatibility), + Default::default() + ); + let status = resident.lock().recorded_status().unwrap(); + let counts = status.observations.for_reason(ObservationReason::Compatibility); + assert_eq!((counts.attempts, counts.successes), (1, 0)); + assert_eq!((counts.attempted_transfers, counts.successful_transfers), (2, 0)); + assert_eq!((counts.attempted_bytes, counts.successful_bytes), (READBACK_BYTES, 0)); + assert_eq!(status.raw_plane_map_attempts, 1); + assert_eq!(board.0.resident_status(), Err(failure)); + assert_eq!(unsafe { (&*board.0.fb_rgb.get()).access_counts() }, Default::default()); + assert_eq!(unsafe { (&*board.0.fb_aux.get()).access_counts() }, Default::default()); + board.0.stop(); + drop(aux_guard); + drop(rgb_guard); }); } @@ -3753,9 +3870,12 @@ fn resident_gpu_gfifo_draws_match_complete_cpu_planes() { cpu_draw(&mut prefix, &mut expected_rgb, &mut expected_aux); write_reg( &rex.0, REX3_DRAWMODE0, - DRAWMODE0_OPCODE_READ | DRAWMODE0_ADRMODE_BLOCK_SH | (1 << 8) | (1 << 9), true, + DRAWMODE0_OPCODE_DRAW | (7 << 2), true, ); - assert_eq!(rex.0.wait_resident_idle(), Err(ResidentError::Unsupported("opcode"))); + assert!(matches!( + rex.0.wait_resident_idle(), + Err(ResidentError::Unsupported(_)) + )); gpu.wait(None).unwrap(); let mut guard = resident.lock(); let prefix_status = guard.status().unwrap(); @@ -3928,10 +4048,103 @@ fn resident_gpu_gfifo_activation_reset_and_fault() { assert_ne!(rex.0.resident_scanout_bindings(&gpu).unwrap().epoch, initial_bindings.epoch); assert_eq!(reset.accepted.sequence, Sequence(0)); assert!(reset.rgb.cpu_shadow_current() && reset.aux.cpu_shadow_current()); + let reset_cpu_rgb = unsafe { (*rex.0.fb_rgb.get())[20 * STRIDE as usize + 10] }; rex.0.start(); - write_reg(&rex.0, REX3_DRAWMODE0, DRAWMODE0_OPCODE_READ | DRAWMODE0_ADRMODE_BLOCK_SH | (1 << 8) | (1 << 9), true); - assert_eq!(rex.0.wait_resident_idle(), Err(ResidentError::Unsupported("opcode"))); - assert_eq!(rex.0.resident_status(), Err(ResidentError::Unsupported("opcode"))); + write_reg( + &rex.0, + REX3_CLIPMODE, + 0xf << CLIPMODE_CIDMATCH_SHIFT, + false, + ); + + let host_mode = dm1_rgb | (1 << 7) | (3 << 8) | (1 << 10); + let host_set = (REX3_BASE | REX3_HOSTRW0) as u64; + let host_go = (REX3_BASE | 0x0800 | REX3_HOSTRW0) as u64; + let read = DRAWMODE0_OPCODE_READ + | DRAWMODE0_ADRMODE_BLOCK_SH + | (1 << 8) + | (1 << 9); + let packed = |red: u32, green: u32, blue: u32| { + let pixel = 0xff00_0000u32 | blue << 16 | green << 8 | red; + (u64::from(pixel) << 32) | u64::from(pixel) + }; + let first = packed(0x11, 0x22, 0x33); + let second = packed(0x44, 0x55, 0x66); + let replacement = packed(0xaa, 0xbb, 0xcc); + + write_reg(&rex.0, REX3_DRAWMODE1, dm1_rgb, false); + write_reg(&rex.0, REX3_WRMASK, 0x00ff_ffff, false); + for (start, end, red, green, blue) in [ + ((10, 20), (11, 20), 0x11, 0x22, 0x33), + ((12, 20), (13, 20), 0x44, 0x55, 0x66), + ] { + write_reg(&rex.0, REX3_COLORRED, red << 11, false); + write_reg(&rex.0, REX3_COLORGRN, green << 11, false); + write_reg(&rex.0, REX3_COLORBLUE, blue << 11, false); + write_reg(&rex.0, REX3_XYENDI, xy(end.0, end.1), false); + write_reg(&rex.0, REX3_XYSTARTI, xy(start.0, start.1), false); + write_reg(&rex.0, REX3_DRAWMODE0, draw, true); + } + + write_reg(&rex.0, REX3_DRAWMODE1, host_mode, false); + write_reg(&rex.0, REX3_XYENDI, xy(13, 20), false); + write_reg(&rex.0, REX3_XYSTARTI, xy(10, 20), false); + write_reg(&rex.0, REX3_DRAWMODE0, read, true); + rex.0.wait_resident_idle().expect("prime resident PIO HostR"); + let pio_first = rex.0.read64(host_go); + assert_eq!((pio_first.status, pio_first.data), (BUS_OK, first)); + rex.0.wait_resident_idle().expect("advance resident PIO HostR"); + let pio_second = rex.0.read64(host_set); + assert_eq!((pio_second.status, pio_second.data), (BUS_OK, second)); + + write_reg(&rex.0, REX3_XYENDI, xy(13, 20), false); + write_reg(&rex.0, REX3_XYSTARTI, xy(10, 20), false); + write_reg(&rex.0, REX3_DRAWMODE0, read, false); + let mut dma = [0; 2]; + assert_eq!(rex.0.dma_read64_bulk(host_go, &mut dma), BUS_OK); + assert_eq!(dma, [first, second]); + let reused = rex.0.resident_status().unwrap(); + assert_eq!(reused.accepted.sequence, Sequence(5)); + assert_eq!(reused.raw_plane_map_attempts, 1); + let hostr = reused.observations.for_reason(ObservationReason::HostRead); + assert_eq!((hostr.attempts, hostr.successes), (1, 1)); + + write_reg(&rex.0, REX3_DRAWMODE1, dm1_rgb, false); + write_reg(&rex.0, REX3_COLORRED, 0xaa << 11, false); + write_reg(&rex.0, REX3_COLORGRN, 0xbb << 11, false); + write_reg(&rex.0, REX3_COLORBLUE, 0xcc << 11, false); + write_reg(&rex.0, REX3_XYENDI, xy(11, 20), false); + write_reg(&rex.0, REX3_XYSTARTI, xy(10, 20), false); + write_reg(&rex.0, REX3_DRAWMODE0, draw, true); + write_reg(&rex.0, REX3_DRAWMODE1, host_mode, false); + write_reg(&rex.0, REX3_XYENDI, xy(11, 20), false); + write_reg(&rex.0, REX3_XYSTARTI, xy(10, 20), false); + write_reg(&rex.0, REX3_DRAWMODE0, read, true); + rex.0.wait_resident_idle().expect("refresh resident HostR snapshot"); + let refreshed = rex.0.read64(host_set); + assert_eq!((refreshed.status, refreshed.data), (BUS_OK, replacement)); + let refreshed_status = rex.0.resident_status().unwrap(); + assert_eq!(refreshed_status.accepted.sequence, Sequence(7)); + assert_eq!(refreshed_status.raw_plane_map_attempts, 2); + let hostr = refreshed_status + .observations + .for_reason(ObservationReason::HostRead); + assert_eq!((hostr.attempts, hostr.successes), (2, 2)); + assert_eq!( + unsafe { (*rex.0.fb_rgb.get())[20 * STRIDE as usize + 10] }, + reset_cpu_rgb, + "HostR must not refresh the board CPU plane", + ); + + write_reg( + &rex.0, + REX3_DRAWMODE0, + DRAWMODE0_OPCODE_DRAW | (7 << 2), + true, + ); + let fault = rex.0.wait_resident_idle().unwrap_err(); + assert!(matches!(fault, ResidentError::Unsupported(_))); + assert_eq!(rex.0.resident_status(), Err(fault)); rex.0.stop(); assert!(!rex.0.is_running()); assert!(rex.0.processor_thread.lock().is_none());