Formula used
With replacement, the success probability is one minus the repeated miss probability. The formula is P = 1 − (1 − p)n. Here, p is one-trial success probability.
Required trials use n = ceil[ln(1 − C) ÷ ln(1 − p)]. Without replacement, the calculator uses an exact hypergeometric probability. Finite spaces therefore avoid duplicate sampling assumptions.
Expected unique samples equal N[1 − (1 − 1/N)n] with replacement. Coverage divides expected unique samples by N. Parallel runtime uses failure-adjusted attempts across active workers.
Success probability: 1 - (1 - p)^n Required trials: ceil(ln(1 - confidence) / ln(1 - p)) Expected successes: n × p Expected unique configurations: N × [1 - (1 - 1/N)^n] Coverage: unique configurations / total configurations Parallel runtime: ceil(expected attempts / workers) × adjusted trial time + overhead
How to use the calculator
- Select the value the calculator should solve.
- Enter confidence, success rate, or known finite successes.
- Choose replacement or without-replacement sampling.
- Describe hyperparameters and their distributions.
- Enter workers, durations, failures, resources, and costs.
- Add scenario trial counts for side-by-side comparison.
- Submit the form and review warnings.
- Copy, print, or export the final results.
Example data
| Input | Example | Purpose |
|---|---|---|
| Acceptable rate | 5% | Assumes five percent of configurations meet the target. |
| Confidence | 95% | Targets a ninety-five percent discovery chance. |
| Workers | 4 | Runs four trials concurrently. |
| Average duration | 30 minutes | Estimates runtime and compute use. |
| Failure rate | 5% | Adds replacement attempts for unstable trials. |
| Worker cost | £0.80 hourly | Estimates total compute spending. |
Interpretation guide
A higher confidence target requires more random trials. Rare acceptable configurations increase trial requirements sharply. Parallel workers reduce wall-clock time, not total compute.
Duplicate risk grows when replacement sampling approaches finite-space size. Without-replacement sampling is preferable for enumerable spaces. Continuous spaces usually need probability-based planning instead.
Cost estimates assume each active worker has the entered hourly cost. Early-stopping savings use a half-duration approximation. Actual schedulers may produce different runtimes.
Frequently asked questions
What does a random search trial represent?
One trial samples a hyperparameter configuration and evaluates it. The evaluation usually trains and validates one model. Failed trials may require replacement attempts.
How many trials are usually enough?
The answer depends on acceptable configuration frequency and confidence. Rare good configurations need many more trials. Use observed results to update estimates.
Why use random search instead of grid search?
Random search explores important dimensions more efficiently in many spaces. Grid search spends trials on every fixed combination. Random distributions also handle continuous parameters naturally.
What is sampling with replacement?
Each draw can repeat a previously sampled configuration. Repetition creates duplicate risk in finite spaces. It remains natural for continuous distributions.
When should without-replacement sampling be used?
Use it when the complete finite search space is enumerable. Every selected configuration remains unique. The calculator then applies exact finite probabilities.
How is the expected best percentile estimated?
It uses the order-statistic approximation one divided by trials plus one. The estimate assumes independent random samples. Lower percentages indicate better expected ranks.
How does early stopping affect runtime?
The calculator assumes stopped trials consume half normal duration. This creates a practical savings estimate. Change the entered rate for sensitivity analysis.
Why can parallel cost remain high?
Parallelism shortens elapsed time but uses several workers simultaneously. Total compute can remain almost unchanged. Hourly worker pricing determines the final estimate.
Can the calculator guarantee a good model?
No probability model can guarantee model quality. Results depend on the defined search space and target. Validation design still determines trustworthy performance.