gpu-sizing

Here are 3 public repositories matching this topic...

amitgambhir / llm-inference-planner

Physics-based LLM capacity planner - size a deployment before provisioning hardware. Roofline model → replica count, cost envelope, and confidence band. Benchmark harness validates estimates against real endpoints and compares configs on latency, cost, and quality.

benchmark nextjs capacity-planning mlops roofline-model fastapi openai-api llm llm-serving vllm llm-inference sglang gpu-sizing

Updated Jun 22, 2026
Python

nemmusu / llm-capacity-planner

Star

GPU memory and concurrency calculator for LLM inference on vLLM and SGLang. For any Hugging Face model: the VRAM for weights and KV cache, which NVIDIA GPUs fit it (RTX to H100, H200, B200, B300), how many concurrent users they serve and the tokens per second per user. No GPU needed; single executables for Windows, Linux and macOS.

cli typescript gpu capacity-planning nvidia vram huggingface kv-cache llm vllm llm-inference sglang vram-calculator tokens-per-second gpu-sizing

Updated Oct 15, 2026
TypeScript

Lowaltitudeflight / llm-private-deploy-handbook

Star

Hands-on handbook for delivering private LLM deployments: environment survey, VRAM estimation with worked examples, vLLM / SGLang / TGI deployment, TTFT and TPOT benchmarking, acceptance testing and troubleshooting fault trees.

Updated Oct 16, 2026
Python

Add this topic to your repo

To associate your repository with the gpu-sizing topic, visit your repo's landing page and select "manage topics."

Learn more