Formula used
Dense layers multiply inputs by every output unit. Convolutions repeat kernel operations across output positions. Counting rules determine whether each MAC means one or two.
| Component | Core estimate |
|---|---|
| Dense | Batch × inputs × outputs × MAC factor |
| Conv2D | Batch × output area × output channels × kernel area × input channels per group |
| Recurrent | Gates × batch × sequence × hidden size × combined input size |
| Attention | Projection operations plus score and value matrix operations |
| Hardware time | Total FLOPs ÷ effective FLOPs per second |
How to use this calculator
Choose the workload and enter global training values. Add every important model layer using matching dimensions. Review assumptions before comparing models or hardware performance estimates.
- Select layer, inference, or training mode.
- Add model layers and verify every output shape.
- Choose the MAC convention and included operations.
- Enter hardware throughput and sustained utilisation.
- Calculate, compare, chart, copy, or export results.
Example data
This example combines convolution and dense classification layers. It uses batches of thirty-two images. Replace these values with your model’s actual dimensions.
| Layer | Inputs | Outputs | Kernel | Batch |
|---|---|---|---|---|
| Conv2D | 224 × 224 × 3 | 224 × 224 × 64 | 3 × 3 | 32 |
| Dense | 4,096 features | 1,000 features | Not applicable | 32 |
| Transformer | 512 tokens | 768 dimensions | 12 heads | 8 |
Understanding FLOPs
FLOPs and FLOP/s
FLOPs describe the arithmetic work required by computation. FLOP/s describes hardware processing speed over elapsed time. A large model can still run quickly on accelerators.
FLOPs and MACs
A MAC combines multiplication with an addition. Some reports count one MAC as two FLOPs. Always compare models using identical counting conventions and inclusions.
Why runtime differs
Memory access, kernel efficiency, and parallelism affect runtime. Equal-FLOP models can show very different measured latency. Benchmarking remains necessary for final deployment and purchasing decisions.
Frequently asked questions
What does this calculator measure?
It estimates arithmetic operations for machine learning workloads. It also estimates parameters, memory, and hardware time. Results remain theoretical until validated with actual benchmarks.
Is one MAC one or two FLOPs?
Both conventions appear in technical reports. This calculator supports either counting method. Use one convention consistently across every model comparison.
Are activation functions included?
Activation operations are optional through the counting controls. Costs differ between simple and transcendental functions. Custom activation layers accept user-defined operation costs directly.
How are training FLOPs estimated?
Training combines forward, backward, and optimizer operations. Epoch totals also use dataset and batch sizes. Checkpoint overhead can represent recomputation or extra processing.
Does lower precision reduce FLOPs?
Precision usually does not change arithmetic operation counts. It changes storage, bandwidth, and hardware throughput. Enter throughput matching the chosen precision for useful timing.
Why is measured speed often lower?
Peak specifications assume ideal kernels and full utilisation. Real workloads include memory and communication bottlenecks. Use sustained utilisation based on trustworthy measured performance.
How does key-value caching help?
Caching reuses previous attention keys and values. Autoregressive decoding then processes fewer new query tokens. Memory usage rises while repeated projection work decreases.
Can grouped convolutions be calculated?
Yes, enter the required number of groups. Input channels must divide evenly across those groups. Depthwise convolution automatically uses one group per channel.
Are FLOPs enough for model selection?
No, FLOPs exclude many practical deployment constraints. Accuracy, latency, memory, energy, and compatibility also matter. Test candidate models on the intended production hardware.