Windows: editor startup blocks for ~26s compiling composite shader with FXC

[Hona]

TL;DR

On Windows, opening a Studio editor can spend 13-29 seconds synchronously compiling composite-video-frame.wgsl through WGPU's default FXC path before the first usable frame reaches the WebView.

A five-run isolated reproduction using Cap's current-main shader, WGPU 25.0.2, the repository's vendored wgpu-hal, DX12, and the same NVIDIA adapter measured:

Compiler	Run 1	Run 2	Run 3	Run 4	Run 5	Average
FXC	24,935.266 ms	26,578.950 ms	28,422.687 ms	22,171.644 ms	28,926.749 ms	26,207.059 ms
DXC, unchanged WGSL	378.561 ms	254.895 ms	204.120 ms	220.595 ms	215.161 ms	254.666 ms

That is a 102.9x speedup and a 99.03% reduction in pipeline creation time without changing the WGSL algorithm.

Retaining the compiled RenderPipeline on the shared device reduced five subsequent reuses below the benchmark's microsecond measurement resolution. The current screenshot prewarm creates a complete RendererLayers and then drops it, so every editor creates the pathological pipeline again.

Environment

• Installed Cap: 0.5.9
• Source investigated and benchmarked against current main: 9bf576051d5743009a8a98c3b63bd6c6b86cd8a5
• Windows: 11, build 10.0.26200
• CPU: Intel Core i7-12800H, 20 logical processors
• GPU: NVIDIA GeForce RTX 3080 Ti Laptop GPU
• Driver: 32.0.15.8097
• Backend: DX12
• WGPU: 25.0.2
• DXC comparison: official Microsoft DirectXShaderCompiler v1.9.2607, July 2026

This is not a WARP/software-adapter result. Cap and the standalone reproduction both selected the discrete NVIDIA adapter.

User-visible timeline

A recent 0.5.9 recording shows that window construction, finalization, and decoding are not the long pole:

Event	Timestamp	Elapsed from stop
Recording stopped; editor requested	22:36:11.485	0 ms
Native editor window shown	22:36:11.599	113 ms
Recording finalization completed	22:36:12.238	752 ms
FFmpeg pre-decoded screen frame 0	22:36:12.448	962 ms
First YUV converter texture initialization	22:36:25.502	14.017 s
Editor instance returned from prewarm	22:36:25.756	14.271 s
First usable frame delivered to the new socket	22:36:26.064	14.579 s

The important interval is:

22:36:12.269  Creating shared YUV converter pipelines
22:36:12.288  Shared YUV converter pipelines created successfully
               <13.214 seconds>
22:36:25.502  Initializing YUV converter textures

Constructor order localizes that gap to the prefix before DisplayLayer construction: the shared composite pipeline, background pipelines, blur pipeline, and trivial frame/notch buffer setup. An isolated per-shader benchmark then identified the composite pipeline:

Current/0.5.9 pipeline	First measured compile
Composite	16,229.760 ms
Image background	16.227 ms
Gradient background	48.105 ms
Background blur	64.835 ms
Cursor	132.474 ms
Mask	76.576 ms
Caption background	24.219 ms

Repeated composite construction remained slow at 13,080.660 ms. A later controlled run measured 26,461.575 ms, consistent with the five-run current-main distribution above.

Critical path

The editor's startup path is:

1. PendingEditorInstances::start_prewarm starts a project-specific prewarm.
2. do_prewarm awaits create_editor_instance_impl.
3. EditorInstance::new starts decoder setup and start_renderer_layers_creation concurrently.
4. The renderer-layers-init OS thread constructs a full RendererLayers and preloads cursor assets.
5. finish_renderer_layers_creation performs an unbounded layers_rx.await.
6. Only after that barrier completes can the editor instance return and the frontend receive a fresh frame.

Relevant code:

• apps/desktop/src-tauri/src/editor_window.rs
• apps/desktop/src-tauri/src/lib.rs
• crates/editor/src/editor_instance.rs
• crates/editor/src/editor.rs
• crates/rendering/src/lib.rs
• crates/rendering/src/composite_frame.rs
• crates/rendering/src/shaders/composite-video-frame.wgsl

instance served from prewarm is slightly misleading in this case. get_or_create can take a pending receiver and then wait for the unfinished prewarm; wait_ms measures that wait, not a cache hit on an already-ready instance.

Why this shader is pathological under FXC

The monolithic fragment shader contains two fixed motion kernels:

• A 21-tap directional kernel.
• A 13-tap radial zoom kernel.

Each tap invokes sample_texture, which contains separate native/downscale/upscale paths and additional texture reads. Each tap also invokes rounded-corner SDF coverage. FXC attempts to unroll and inline these constant loops into a large shader program before creating the DX12 pipeline state object.

This is DX12's synchronous behavior: pipeline creation does not return until shader compilation and PSO creation complete. Microsoft documents that behavior here:

• New in D3D12 - background shader optimizations
• HLSL for statement compiler controls

A controlled variant that removed only the motion-blur section reduced pipeline compilation from 26,461.575 ms to 408.791 ms. That experiment changes motion-blur output and is diagnostic only, not the proposed fix.

A second experiment moved the exact 21/13 tap count into the existing unused uniform lane to encourage a rolled loop. FXC rejected it:

warning X3570: gradient instruction used in a loop with varying iteration, attempting to unroll the loop
error X3511: unable to unroll loop ... use the [unroll(n)] attribute

The tap body uses implicit texture gradients and fwidth for rounded-corner coverage, so a naive dynamic-loop rewrite is not portable through the current WGSL -> Naga -> FXC path.

DXC result with unchanged WGSL

WGPU 25 supports Dx12Compiler::DynamicDxc. Its own API documentation describes FXC as old, slow, and unmaintained, and DXC as new, fast, and maintained.

Using Microsoft's official July 2026 dxcompiler.dll and dxil.dll, with the current-main WGSL unchanged, reduced the five-run mean from 26,207.059 ms to 254.666 ms.

The WebView2-bundled DXC DLLs were also tested but caused RequestDeviceError(DeviceLost) and are not included in any result. A real implementation should bundle and sign a supported official DXC pair rather than borrow WebView2's private runtime files.

Quality verification

I rendered a deterministic 360-frame RGBA stress sequence at 640x360 and 60 fps using current-main composite-video-frame.wgsl:

• Frames 0-179: directional motion blur.
• Frames 180-359: radial zoom blur.
• Moving high-frequency grid/checker source.
• Bilinear resampling.
• Rounded/squircle corners.
• Shadow and border.
• Current-main color grading and grain.

Metrics were calculated on raw RGBA frames before H.264 encoding:

Comparison	PSNR	SSIM
FXC vs retained FXC pipeline	infinite; SHA-256 identical	1.000000
FXC vs unchanged WGSL compiled by DXC	86.553432 dB	0.999999

The FXC and DXC raw streams are not byte-identical because the compilers produce slightly different floating-point results. The difference is not visually meaningful: SSIM rounds to 0.999999, and an amplified 32x difference video remains almost entirely black.

The unchanged-WGSL DXC result is therefore semantically equivalent and visually indistinguishable, but it should not be described as byte-for-byte identical to FXC. Retaining the same compiled pipeline is byte-identical.

Comparison, individual, raw, and 32x-difference artifacts have been generated and can be attached separately.

Repeated work and ineffective prewarm

The app-wide screenshot renderer prewarm:

• Starts asynchronously after main-window-ready.
• Constructs a temporary full RendererLayers.
• Performs a real render.
• Drops the layer and pipeline objects when it returns.

The editor later constructs a second full RendererLayers on the same device. The driver cache did not make repeated FXC creation cheap; measured repeated compiles remained in the 13-29 second range.

There is sharing inside one RendererLayers: display, camera, and camera-only use one shared composite pipeline. There is no device-scoped sharing across screenshot and editor instances.

Cap's explicit WGPU pipeline cache also does not activate on the normal desktop shared-device path:

• CompositeVideoFramePipeline uses a cache only when wgpu::Features::PIPELINE_CACHE is enabled.
• gpu_context.rs requests the desktop shared device with required_features: wgpu::Features::empty().
• Other render-pipeline descriptors use cache: None.
• No Cap shader_cache.bin was produced on the tested installation.

Proposed remediation

1. Bundle and select official DXC on Windows

Configure the Windows WGPU instance with a supported official Dx12Compiler::DynamicDxc pair. Keep a tested FXC fallback if loading DXC fails.

Expected result from the isolated five-run measurement: approximately 255 ms cold composite pipeline creation instead of 26.2 seconds.

2. Retain device-scoped immutable pipelines

Move the composite pipeline, and potentially other immutable pipelines, into SharedGpuContext or another device-owned cache and pass shared Arcs into each RendererLayers.

The existing prewarm can then populate and retain the real objects instead of compiling and dropping them. Five retained-pipeline clones measured below 0.001 ms each after the initial compile.

3. Enable and broaden pipeline caching

Request wgpu::Features::PIPELINE_CACHE when supported by the selected adapter, qualify persisted data by adapter/driver/shader version, and apply the cache to more than the composite pipeline.

This is secondary to retaining live objects and using DXC. Cache loading must safely fall back after driver or shader changes.

4. Add startup-stage instrumentation and a Windows regression benchmark

Record independent timings for:

• Recording readiness/finalization.
• Shared GPU acquisition and adapter/compiler choice.
• Segment/decoder completion.
• Each RendererLayers constructor.
• Cursor preload.
• First render completion and first WebSocket frame.

A focused Windows benchmark should run current-main composite pipeline creation in a fresh process and report compiler, adapter, mean, min, and max. The existing weekly performance workflow uses Windows runners, but its current fixtures/gates did not expose this machine-specific cold pipeline latency.

Suggested acceptance criteria

• Five-run Windows composite pipeline mean below 500 ms on the affected NVIDIA machine.
• Editor startup no longer waits on a repeated FXC composite compile.
• DXC DLL loading failure falls back cleanly and is visible in logs.
• FXC and DXC stress renders meet an agreed quality threshold, for example SSIM >= 0.99999 against the current path.
• Screenshot editor, video editor, export preview, and export all use the same retained device-scoped pipeline implementation.
• Existing macOS and software-adapter paths remain unchanged unless separately measured.

Scope note

This issue is independent of the Windows camera-enumeration leak in #2129. The shader stall reproduces in a fresh process before camera-resource accumulation can explain it.

[linear]

CAP-790