Skip to content
#

flash-decoding

Here are 6 public repositories matching this topic...

Language: All
Filter by language

CuTileForge v3.0.0: Native tile-fusion compiler & universal quantization engine on NVIDIA CuTile. Zero-ATen CUDA-graph decode at 172 tok/s, 2,377 tok/s batch serving, 13 model families, 70B hybrid on a single 32 GB GPU (sm_89 ↔ sm_120).

  • Updated Nov 21, 2026
  • Python

C++/CUDA inference engine for Qwen2.5-Coder-0.5B, written from scratch: INT4 GEMV/GEMM kernels, Flash-Decoding, a 24-layer decode engine and lossless speculative decoding. Verified against Hugging Face; benchmarked against bandwidth ceilings, PyTorch and llama.cpp.

  • Updated Oct 11, 2026
  • Python

Add this topic to your repo

To associate your repository with the flash-decoding topic, visit your repo's landing page and select "manage topics."

Learn more