GPU memory breakdown
| Memory component | Unsharded | Per GPU | Share |
|---|---|---|---|
| Total including reserve | 0 GiB | 0 GiB | 100% |
Fit and scaling details
| Minimum approximate GPU count | 1 |
|---|---|
| Maximum estimated micro-batch | 0 |
| Memory shortfall | 0 GiB |
| Global batch validation | Not checked |
| Trainable parameters | 0 |
Optimisation suggestions
Precision and tuning comparison
| Scenario | Weight memory | Estimated change |
|---|
Step-by-step calculation
Calculation details will appear here.
Formula used
Weight memory = parameters × bytes per weight Gradient memory = trainable parameters × bytes per gradient Optimiser memory = trainable parameters × optimiser state bytes Activation memory ≈ batch × sequence × hidden size × layers × bytes × multiplier KV cache = 2 × layers × batch × total sequence × KV heads × head dimension × bytes Required VRAM = sharded components + activations + inputs + buffers + overhead + reserve
This calculator separates persistent tensors from temporary tensors. It then applies precision, sharding, offloading, and safety settings. The result remains an engineering estimate, not a framework guarantee.
How to use the calculator
- Select a model preset or enter parameters manually.
- Choose training, fine-tuning, or inference mode.
- Set precision, optimiser, batch, and sequence values.
- Add quantisation, sharding, and offloading choices.
- Select GPU capacity and review the fit result.
- Export the breakdown for planning or documentation.
Start with measured values from your actual model configuration. Keep a safety margin for allocator fragmentation and temporary kernels. Validate final settings with a small real workload run.
Example data
| Scenario | Parameters | Method | Weight precision | Typical focus |
|---|---|---|---|---|
| BERT-base fine-tuning | 110 million | Full fine-tuning | FP16 | Gradients and Adam states |
| 7B language model | 7 billion | QLoRA | NF4 | Quantised weights and activations |
| 13B text generation | 13 billion | Inference | INT8 | Weights and KV cache |
| Vision model training | 86 million | Full training | BF16 | Image activations and workspace |
Precision memory guide
| Format | Approximate bytes | Common use |
|---|---|---|
| FP64 | 8 | Special numerical workloads |
| FP32 or TF32 storage | 4 | Master weights and stable training |
| FP16 or BF16 | 2 | Mixed-precision training |
| INT8 | 1 | Compressed inference |
| INT4 or NF4 | 0.5 plus metadata | Large-model inference and QLoRA |
Frequently asked questions
Why can actual GPU memory exceed this estimate?
Framework allocators reserve memory and create temporary tensors. Kernel selection can also change workspace requirements. Always test the final configuration on target hardware.
Does parameter memory equal total training memory?
No. Training also stores gradients, optimiser states, activations, and workspaces. Those components can exceed the model weights.
How does LoRA reduce memory?
LoRA trains small low-rank matrices while base weights remain frozen. This reduces gradient and optimiser memory. Activation memory may still remain significant.
Why does sequence length matter so much?
Longer sequences increase activations and KV cache sizes. Standard attention buffers may grow quadratically. Flash Attention can reduce that temporary memory.
What does ZeRO Stage 3 change?
It shards parameters, gradients, and optimiser states across data-parallel workers. Communication overhead and runtime peaks still matter.
Should I include FP32 master weights?
Many mixed-precision training setups keep FP32 master copies. Disable them only when your optimiser and framework avoid them.
Is GPU capacity measured in GB or GiB?
Manufacturers and tools may use different unit conventions. This calculator treats selected GPU capacity as GiB. Decimal output units remain available separately.
How much safety margin is reasonable?
Ten to twenty percent is a common planning range. Dynamic shapes and variable sequence lengths may need more.
Can this calculate maximum batch size?
It provides an approximate batch limit from scalable activation memory. Real limits depend on kernels, graph shape, and allocator behaviour.