Formula used
Parameter memory = parameter count × bytes per weight.
Gradient memory = trainable parameters × bytes per gradient.
Optimizer memory = trainable parameters × optimizer-state multiplier × bytes per state.
Activation memory = stored activation elements × effective batch size × bytes per activation.
KV-cache memory ≈ 2 × layers × batch × tokens × hidden size × KV-head ratio × bytes.
Total memory = weights + activations + gradients + optimizer states + master weights + tensors + cache + workspace + overhead.
How to use
- Choose inference, training, or fine-tuning mode.
- Load a preset or build each model layer.
- Select numerical precision and optimizer settings.
- Enter transformer, batch, and GPU details.
- Calculate and review component-level memory usage.
- Export the summary as CSV, PDF, or print.
Example configurations
| Model | Typical parameters | Common workload | Main memory pressure |
|---|---|---|---|
| Small MLP | 100K–10M | Tabular classification | Weights and optimizer states |
| ResNet-style CNN | 20M–100M | Image training | Feature-map activations |
| BERT-style encoder | 100M–400M | Sequence training | Activations and optimizer states |
| GPT-style decoder | 1B+ | Generation | Weights and KV cache |
| U-Net | 10M–100M | Segmentation | Skip-connection activations |
Frequently asked questions
Why does training use more memory than inference?
Training stores gradients, activations, optimizer states, and often master weights. Inference usually stores only weights, active tensors, workspace, and optional KV cache.
What is the largest training memory component?
It depends on architecture and batch size. Optimizer states dominate large parameter models, while activations dominate deep CNNs and long-context transformers.
How does mixed precision reduce memory?
FP16 or BF16 halves many tensor sizes compared with FP32. Some training systems still keep FP32 gradients, optimizer states, or master weights.
What does gradient checkpointing change?
It stores fewer forward activations and recomputes them during backpropagation. This reduces memory while increasing computation time.
Why can measured GPU usage exceed this estimate?
Framework allocators reserve memory and kernels need temporary workspaces. Fragmentation, libraries, graph compilation, and communication buffers also add overhead.
How is KV-cache memory calculated?
The estimate scales with layers, batch size, token count, hidden size, KV-head ratio, and cache precision.
What is ZeRO memory sharding?
ZeRO partitions optimizer states, gradients, and parameters across data-parallel devices. Higher stages reduce per-device memory but increase communication complexity.
Can this calculator find maximum batch size?
Yes. It separates fixed parameter memory from batch-scaled tensors, then estimates the largest batch fitting the chosen capacity.
Are preset values exact?
No. Presets are planning baselines. Use exact model summaries or profiler measurements for production deployment decisions.
Should GiB and GB be treated equally?
No. This calculator reports binary units using 1 GiB = 1,073,741,824 bytes. Vendor labels may use decimal units.