Repository navigation
🚨 Metal: accelerate 1D IM2COL with cooperative tiling - #1057
Merged
Merged
Conversation
0xShug0
changed the title
cunba-ai
pushed a commit
to cunba-ai/audio.cpp
that referenced
this pull request
Oct 28, 2026
…(Qwen3/YuE2/ACE-Step/MiniMax cuda_perf) + Metal IM2COL + Gradium Phonon + Drama3/Ming3-Tiny speedups Upstream commits 12cd5bb..eefd932 (17): - multilingual Granite TurboCTC checkpoint (0xShug0#1035) - Qwen3 TTS optimization controls + extended cuda_perf preset (0xShug0#1055); preset renamed perf_first - VoxCPM2 optimized preset renamed cuda_perf - YuE2 performance controls + cuda_perf preset (0xShug0#1056) - Zerodot1/ming3 parallel disk reads (0xShug0#1042) - Metal: 1D IM2COL cooperative tiling (0xShug0#1057) - Zerodot1/arch linux amd installer (0xShug0#1058) - opt-in plain RMS norm for timestep embeddings (0xShug0#1061) - Gradium Phonon TTS on PocketTTS runtime (0xShug0#1062) - Drama3 sampling optimized, tower decode on-device (0xShug0#1063) - parallel expert reads on Apple without OpenMP (0xShug0#1064) - UI models + LatentSync panel + models-folder history (0xShug0#1066) - Drama3 codec batching + overlap decode (0xShug0#1065); Gemma scales precomputed (0xShug0#1067) - Drama3 decode perf controls + F16 KV default (0xShug0#1068) - Ming3 Audio Tiny 2x faster, 20% less VRAM (0xShug0#1069) - ACE-Step perf controls + cuda_perf preset (0xShug0#1073) - MiniMax Music3 perf controls + compact output head (0xShug0#1077) # Conflicts: # src/models/qwen3_tts/talker.cpp # tests/unittests/test_decoder_packed_projections.cpp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replace tiny per-position Metal IM2COL threadgroups with cooperative 32x32 time/patch tiles, coalescing both input reads and output writes. This improves convolution-heavy audio decoders without changing their graphs or arithmetic.
Only three GGML Metal files change. The 2D path and other backends retain their existing implementation. No profiling code, debug gates, framework/model changes, or reports are included.
Validation
Apple M4 Pro, Metal; one warmup and medians of three measured runs. Each before/after pair uses identical inputs and settings.
Request coverage: 249-character TTS requests for KittenTTS2, Kokoro, PocketTTS, SanoTTS, Chatterbox, CosyVoice3, and IndexTTS2; 81-character cloning requests for BreezeTTS2, Qwen3-TTS, VoxCPM2, and VibeVoice; a 25.6-second music request for YuE2. Weights: Q4 KittenTTS2/BreezeTTS2, F32 SanoTTS, Q8 for the remaining models, with F16 VAE for YuE2. All paired WAVs are byte-identical.
SanoTTS component timings are only a few milliseconds and varied on repeats. The changes of 1.3% or less also do not establish a meaningful end-to-end gain.
Affected-component timings: Chatterbox HiFT graph 950 -> 290 ms, CosyVoice3 HiFT graph 1212 -> 368 ms, IndexTTS2 BigVGAN 6111 -> 4022 ms, and YuE2 VAE 4229 -> 932 ms.
Added
tests/unittests/test_metal_im2col.cpp: 1,372 cases, each executed twice with changed input and poisoned output buffers. Every output is checked against frozen SHA-256 fingerprints captured from the original Metal kernel atc8365f39, plus independent bit-exact scalar checks. Coverage includes F32/F16, tile boundaries, padding, stride, dilation, batching, offset views, 2D, fast-1D lowering, and production HiFT shapes. A deliberately corrupted baseline correctly fails.The test is enabled only by
ENGINE_BUILD_EXTENDED_TESTS=ONwith Metal support; unit-test-only builds exclude it. Run withctest --test-dir build -R "^metal_im2col_test$" --output-on-failure. Optional--benchmarkreports production-shape timings without timing-based pass/fail thresholds.