Skip to content

Switch to the shared Sdcb.SimdOnnx engine - #4

Merged
sdcb merged 10 commits into
mainfrom
feature/simdonnx
Oct 16, 2026
Merged

sdcb merged 10 commits into
mainfrom
feature/simdonnx

Conversation

@sdcb

@sdcb sdcb commented Oct 14, 2026 •

Copy link
Copy Markdown
Owner

Switches the inference engine to the shared Sdcb.SimdOnnx (pinned 0.1.0-preview.3, ProGet test feed). The engine grew out of this repo, so the in-repo interpreter, kernels and Vulkan/Metal backends move there. The GCDR-specific glue stays here: pixel shaders (bgr2nhwc/div6/nhwc2bgr), the fused GPU pipelines and PixelOps.

Performance (interleaved A/B, Ryzen 9 5950X + Arc B580, Sdcb.SimdGcdrNet.Benchmark e2e)

case main ms this PR ms ratio peak WS MB
cpu 1600x1152 t=1 1229.9 1002.2 0.81 303/303
cpu 1600x1152 t=4 517.6 458.4 0.89 307/311
cpu 1600x1152 t=16 265.6 248.3 0.93 339/343
vulkan 1600x1152 16.6 16.6 1.00 173/173
cpu 3200x2304 t=16 835.3 730.6 0.87 903/909
vulkan 3200x2304 65.2 64.9 1.00 205/206

Tests: 74 passed and 30 skipped (device-specific: AVX-512, NEON, Metal); they also pass against the published package. Script: SimdOnnx/tools/ab-gcdr.ps1.

Before merge (TODO)

  • NuGet. Publish Sdcb.SimdOnnx to nuget.org, drop the simdonnx-preview source from nuget.config, and bump the pin.
  • Metal. Not run on Apple hardware in this change.
  • Android. AndroidBench was not built locally (no android workload installed).
  • Flaky Vulkan tests. The cause was test interference: NoFuse_MatchesFused set GCDRNET_NOFUSE for the whole process, and the ConvDot test flipped ConvDot on the shared device. Both are fixed: the test now uses CompileOptions.Fuse, and the ConvDot test runs on a private device. Result: 0 failures in 35 parallel full-suite runs.
  • Device loss. Enhance retries the image once on a fresh device. The test is opt-in: SIMDONNX_TEST_DEVICELOSS=1.
  • CI. This repo has no CI workflow. Its fixtures are exercised by the SimdOnnx test workflow.

sdcb and others added 4 commits October 14, 2026 19:55
…xtensions (2.0.0)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…e NOFUSE / shared-device ConvDot toggling (fixes flaky Vulkan byte-exact tests), opt-in device-loss test
@sdcb
sdcb force-pushed the feature/simdonnx branch from bcc0436 to a2a0bc9 Compare October 14, 2026 11:55
sdcb and others added 6 commits October 15, 2026 13:08
…tile-read slack included); fused pipeline opts into the staged SG16 tensor-core convs (CompileOptions.StagedCoopConv); CI builds against sdcb/SimdOnnx main from source

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…extensions, PixelOps, fused pipelines) and points to Sdcb.SimdOnnx for the engine; migration note for the engine types that moved to Sdcb.SimdOnnx.*; package description; rk3399 doc links the NEON kernels in SimdOnnx

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s and packs against the pinned package (what ships: the nupkg now depends on exactly [0.1.0-preview.4]) and keeps one job on sdcb/SimdOnnx main from source; NU5104 accepted while the engine is a preview

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…org; wasm doc links the PackedSimd kernels in SimdOnnx

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sdcb
sdcb marked this pull request as ready for review October 16, 2026 11:20
@sdcb
sdcb merged commit 9be6a58 into main Oct 16, 2026
36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant