Skip to content
#

inference-optimization

Here are 545 public repositories matching this topic...

Hands-on LLM algorithms and systems with PyTorch, Triton & CUDA.《大模型算法与系统教程》:模型架构与训练、后训练、推理加速、性能与算子优化,配套 Notebook 练习、答案解析、测试与实验。

  • Updated Nov 16, 2026
  • Jupyter Notebook

LLM infrastructure cost reduction via NUMA-aware weight banking: 147 t/s (8.8x stock llama.cpp) on refurbished enterprise POWER8. Self-hosted inference, no cloud APIs. Part of the Proof of Physical AI stack.

  • Updated Nov 7, 2026
  • Python

A curated list of strategies, tools, papers, and resources for reducing LLM token costs and improving efficiency in production.

  • Updated Nov 15, 2026
  • Shell
local-llm-tuning-guide

A decision library for running local LLMs on multi-GPU machines without NVLink: parallelism, communication, quantization, speculative decoding, KV cache. Every conclusion carries its evidence grade and reproducible conditions.

  • Updated Nov 12, 2026
  • Python

Add this topic to your repo

To associate your repository with the inference-optimization topic, visit your repo's landing page and select "manage topics."

Learn more