Skip to content

Interop slices, formats and GPU waits; enable2-only Vulkan; GPU skybox; mirror flips; fades on own frames - #81

Open
feimos-dev wants to merge 19 commits into
mainfrom
interop-mirror-textures
Open

feimos-dev wants to merge 19 commits into
mainfrom
interop-mirror-textures

Conversation

@feimos-dev

@feimos-dev feimos-dev commented Nov 20, 2026 •

Copy link
Copy Markdown
Owner

Graphics interop (OpenGL and Vulkan copied through D3D11)

  • Array slices. The layer shown is chosen the same way the native compositors choose it. OpenGL: SubmitWithArrayIndex's slice, else the eye's for Submit_GlArrayTexture (read with glFramebufferTextureLayer). Vulkan: SubmitWithArrayIndex's slice, else VRVulkanTextureArrayData_t::m_unArrayIndex for Submit_VulkanTextureWithArrayData.

  • OpenGL renderbuffers and multisampled textures. Submit_GlRenderBuffer and GL_TEXTURE_2D_MULTISAMPLE sources are copied too. A multisampled one goes through a resolving blit, which can't also flip rows or change the format: that copy stays in GL row order, and the bounds flip it instead (vMin > vMax, which the D3D11 shader pass applies); a format with no D3D11 format of the same texels is refused. A single-sampled renderbuffer is copied as a texture is. Copies are keyed by name and kind, because a renderbuffer and a texture can share a name. A name of another kind than the submit says isn't read as the texture bound before it.

  • More formats. The formats now come from the compositors' format table. Two new functions in FormatTable do the mapping, both unit-tested:

    • SameTexelsIn finds another API's format with the same texels.
    • ConvertibleColourIn gives a format a GL blit can convert into without losing range: 8-bit RGBA with the same sRGB-ness, or 16-bit float RGBA.
    • Vulkan uses only identical texels, since a copy can't convert; OpenGL can also use the converting blit.
  • GPU-side waits for Vulkan. Two ID3D11Fences are created with D3D11_FENCE_FLAG_SHARED and imported into the game's device as semaphores (vkImportSemaphoreWin32HandleKHR, VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_D3D11_FENCE_BIT). Values are passed through VkD3D12FenceSubmitInfoKHR. Each frame:

    1. D3D11 Signals and Flushes.
    2. The Vulkan copy waits for that value, then signals its own.
    3. D3D11 Waits for the copy's value.

    This replaces the CPU fence wait of up to 100 ms per eye per frame. If either side lacks the fences or semaphores, the CPU wait stays as before, and the choice is logged once. The D3D11 device's context is made multithread-protected, as the game's submits and overlays both use it.

  • A ring of command buffers. Four command buffers and fences are used in turn (two eyes, two frames running), so an eye's copy doesn't wait for the other's. Copies are kept per image and array layer, up to 32.

Vulkan device extensions

  • GetVulkanDeviceExtensionsRequired now uses pPhysicalDevice, but only for a device that GetOutputDevice returned. The conformance suite passes arbitrary pointers, and a pointer that isn't a physical device can't be told apart from one.
  • On the interop path it adds VK_KHR_external_semaphore and VK_KHR_external_semaphore_win32 when the device has both; the instance list has VK_KHR_external_semaphore_capabilities.
  • On the runtime path it leaves out runtime-listed extensions the GPU lacks, and logs them once. Asking for a missing extension makes vkCreateDevice fail; leaving it out risks only a failed session, which has a fallback.

Vulkan on runtimes with XR_KHR_vulkan_enable2 alone

  • When XR_KHR_vulkan_enable isn't offered, OpenLamina enables XR_KHR_vulkan_enable2. The xr_ext wrappers then go through xrGetVulkanGraphicsDevice2KHR and xrGetVulkanGraphicsRequirements2KHR.
  • The temporary device is made through xrCreateVulkanInstanceKHR and xrCreateVulkanDeviceKHR.
  • On Windows with D3D11, game frames go through the interop. On a runtime with XR_KHR_vulkan_enable2 and no D3D11 (Linux), the game's own instance and device are bound without xrCreateVulkanDeviceKHR.
  • New Linux conformance run: scripts/gfx_vk_enable2.json with expect/gfx_vk_enable2.json.

Six-face skybox on the GPU (D3D11)

  • New Compositor::InvokeCubemap. In D3D11 it:
    1. copies (or resolves) the six faces into one 6-slice array texture,
    2. draws the lat-long image into a swapchain of its own with a new pixel shader (ps_cube). The shader mirrors CubeFacesToEquirect's face selection and orientation exactly, and handles sRGB the same way as the existing pass.
  • The other APIs keep the CPU readback-and-join (the GPU pass is only tried for D3D11 textures). If the pass can't be set up, or the faces differ in device, size or format, D3D11 falls back to it too. The pass filters linearly where the CPU join takes the nearest texel.
  • Skyboxes are only shown in own frames, which run only on D3D11.

D3D11 mirror texture: flipped bounds

MirrorUpdate now applies flipped bounds by copying one row or column at a time, since CopySubresourceRegion can't flip. When both directions flip, the copy goes through a texture of the area's size, kept for the next frame. A multisampled frame is resolved first, as before.

Fades on own frames

Own frames carry the fade over the scene, the same way the game's frames do. A visible fade keeps own frames running.

Checked on Linux

scripts/ci/build-linux.sh --tests x64 (including the new unit tests), the stand-in runtime ctest, scripts/ci/conformance-linux.sh x64 and abort_sites.py --check all pass.

ps_cube is compiled at runtime with D3DCompile. If it fails to compile, skyboxes fall back to the CPU path.

Fixed after the GPU checks

  • Vulkan interop copies take turns in a ring of command buffers. With one command buffer and fence, the right eye's copy waited on the CPU for the left eye's, which was still queued behind the game's work, so the GPU-side wait saved nothing (3.7 ms against 3.5 ms in the busy case below). Now four command buffers and fences are used in turn (two eyes, two frames running). A copy waits only for its slot's copy two frames ago, and any change that frees the copies' images or the command buffers waits for all of them first.
  • SubmitWithArrayIndex through the interop. The slice was applied to the game's texture by the interop's copy, and again by the D3D11 compositor to the copy, which holds only that slice: the submit answered InvalidTexture. The compositor now reads the copy's only slice.
  • Vulkan array layers at or past m_unArraySize are refused; the copy's semaphore wait covers all its commands.

Checked on a GPU

On Windows with an NVIDIA GPU, against the stand-in runtime, with new conformance cases (in CI; the OpenGL and Vulkan ones skip there, as CI has no driver for them). The Vulkan cases also ran with the Khronos validation layer on, which reported nothing.

  • skybox-6 matches expect/skybox.json and the log has no "joined on the CPU": the GPU pass drew it.
  • Six-face skybox against the CPU join: the stand-in now reads an 8 x 4 grid of samples from lat-long layers, and the GPU pass's image matched the CPU join's (the same case with the old code) in every cell, for skybox-6 and skybox-1. The grids are pinned in expect/skybox.json.
  • Frames cleared to a new colour each frame on the game's busy Vulkan queue (gfx-vk-interop-gpu-wait "painted"): the last frame's colour shows. A build without D3D11's wait showed the frame before's in one eye, now and then, so the case can catch that.
  • A Vulkan game on a runtime with D3D11 alone whose device has the extensions GetVulkanDeviceExtensionsRequired names (gfx-vk-interop-gpu-wait): the log says "waited for on the GPU (shared fences)", and frames show. Frame time, as the Submit calls' time per frame over 300 frames (--case-arg time=300), with the game's queue still clearing a 4096 x 4096 image 40 times when it submits (--case-arg busy=40): 0.95 ms with the GPU wait against 3.49 ms with the CPU wait (--case-arg plain=1). With the GPU idle, the GPU wait costs a little more: 0.45 ms against 0.17 ms, for the fence signal and flush per copy.
  • The same on a device without them (gfx-vk-interop): the log says "on the CPU", and frames show.
  • Interop array textures (gfx-gl-interop-odd, gfx-vk-interop-odd): Submit_GlArrayTexture, Submit_VulkanTextureWithArrayData and SubmitWithArrayIndex each show the right slice. (SubmitWithArrayIndex only since the fix above.)
  • Interop GL renderbuffers, multisampled renderbuffers and multisampled textures (gfx-gl-interop-odd, with another texture left bound in the context, as games have): upright; a renderbuffer with flipped bounds is upside down, as it should be.
  • Formats that used to be refused (gfx-gl-interop-formats, gfx-vk-interop-formats, under a runtime with D3D11 alone and RGBA8 sRGB swapchains only): GL_RGB8, GL_R11F_G11F_B10F, GL_RGBA16F, GL_RGB10_A2, Vulkan B10G11R11_UFLOAT, R16G16B16A16_SFLOAT, A2B10G10R10_UNORM and B8G8R8A8_SRGB all show the pattern.
  • GetMirrorTextureD3D11 with flipped bounds (mirror-flipped): the mirror is the right way round for bounds flipped in v, in u and in both.
  • A D3D11 game that stalls in a fade to opaque black with ownFrames on and nothing else shown (own-frames-fade): own frames keep going, each with the fade over the game's last frame.

The fade, mirror and interop array, renderbuffer and multisampled cases also ran against main's code: each failed there, but for the Vulkan array data one.

Not in this change

  • D3D12 and Vulkan mirrors, and render-model textures on other APIs: OpenVR has no such entry points.
  • GPU skybox passes for D3D12, Vulkan and GL: skyboxes are only shown in own frames, which run only on D3D11.

@feimos-dev feimos-dev changed the title Vulkan on enable2-only runtimes; fades on own frames Interop slices, formats and GPU waits; enable2-only Vulkan; GPU skybox; mirror flips; fades on own frames Nov 20, 2026
@feimos-dev
feimos-dev marked this pull request as ready for review November 20, 2026 23:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants