GPU Calculator

Model size + quant → how much VRAM you need, which GPUs fit.

MODEL

PARAMETERS (BILLIONS)

WEIGHT PRECISION

KV CACHE PRECISION

ESTIMATED TOTAL MEMORY

Weights

KV cache

Training (grads + optimizer + activations)

Runtime overhead

GPU	Used	Tokens/s	TTFT	Price

GPU

COUNT

SYSTEM RAM (GB)

RAM BANDWIDTH

How is this calculated?

KV cache comes from the model's real architecture (HF config.json): layers × KV heads × head dim, with MLA, sliding-window and hybrid (linear-attention) layers handled separately.
For MoE models memory uses total parameters, speed uses active parameters.
Generation is memory-bandwidth bound: each token reads the active weights + KV (70% efficiency), capped by compute at 40% MFU. TTFT is the time to prefill the whole prompt.
Multi-GPU: tensor parallel over NVLink / Infinity Fabric (×0.85 scaling); layer split over PCIe (adds memory, not speed). 92% of each GPU is usable; on Apple, 75% of RAM.
Runtime overhead is 0.75 GB + 5% of weights. Real usage varies ±10–20% by engine (vLLM, llama.cpp, Ollama).