Regret Curve Calculator

Dataset

Required columns are step, algorithm, reward, and optimal_reward. Optional columns support expected regret, baselines, actions, and repeated runs.

CSV, TSV, or semicolon-separated data

step,algorithm,reward,optimal_reward,expected_reward,baseline_reward,chosen_action,optimal_action,run
1,Epsilon-Greedy,0.55,0.90,0.58,0.50,A,B,1
2,Epsilon-Greedy,0.72,0.92,0.70,0.52,B,B,1
3,Epsilon-Greedy,0.68,0.91,0.69,0.54,B,B,1
4,Epsilon-Greedy,0.80,0.93,0.78,0.55,B,B,1
5,Epsilon-Greedy,0.76,0.94,0.79,0.57,A,B,1
6,Epsilon-Greedy,0.86,0.95,0.84,0.58,B,B,1
1,Epsilon-Greedy,0.50,0.90,0.56,0.50,A,B,2
2,Epsilon-Greedy,0.69,0.92,0.68,0.52,B,B,2
3,Epsilon-Greedy,0.71,0.91,0.70,0.54,B,B,2
4,Epsilon-Greedy,0.77,0.93,0.77,0.55,B,B,2
5,Epsilon-Greedy,0.81,0.94,0.80,0.57,B,B,2
6,Epsilon-Greedy,0.88,0.95,0.85,0.58,B,B,2
1,UCB,0.61,0.90,0.62,0.50,A,B,1
2,UCB,0.75,0.92,0.74,0.52,B,B,1
3,UCB,0.79,0.91,0.78,0.54,B,B,1
4,UCB,0.85,0.93,0.84,0.55,B,B,1
5,UCB,0.89,0.94,0.88,0.57,B,B,1
6,UCB,0.92,0.95,0.91,0.58,B,B,1
1,UCB,0.58,0.90,0.60,0.50,A,B,2
2,UCB,0.73,0.92,0.72,0.52,B,B,2
3,UCB,0.81,0.91,0.79,0.54,B,B,2
4,UCB,0.86,0.93,0.85,0.55,B,B,2
5,UCB,0.90,0.94,0.89,0.57,B,B,2
6,UCB,0.93,0.95,0.92,0.58,B,B,2
1,Thompson Sampling,0.64,0.90,0.65,0.50,B,B,1
2,Thompson Sampling,0.78,0.92,0.77,0.52,B,B,1
3,Thompson Sampling,0.83,0.91,0.82,0.54,B,B,1
4,Thompson Sampling,0.88,0.93,0.87,0.55,B,B,1
5,Thompson Sampling,0.92,0.94,0.91,0.57,B,B,1
6,Thompson Sampling,0.94,0.95,0.93,0.58,B,B,1

Upload CSV

Calculation options

Regret basis

Curve type

Objective direction

Comparator or baseline

Baseline algorithm name

Repeated-run aggregation

Discount factor

Moving-window size

Confidence level

Smoothing method

Smoothing window

Exponential alpha

Reward normalisation

Missing-value handling

Negative regret handling

Burn-in rows per run

Evaluation start step

Evaluation end step

Algorithms to include

Regret threshold

Show confidence bands

Show threshold line

Highlight best algorithm

Chart options

Chart title

X-axis label

Y-axis label

Legend position

Line width

Marker size

Chart height

Use logarithmic Y-axis

Formula used

Instantaneous regret: r_t = R_t^* − R_t

Cumulative regret: R_T = Σ_t=1^T r_t

Average regret: R̄_T = R_T ÷ T

Discounted regret: D_t = γD_t−1 + r_t

Instantaneous regret measures one decision. Cumulative regret adds losses across all decisions. Lower growth usually indicates better long-term learning.

How to use

Paste data or upload a CSV file.
Select the regret definition and objective direction.
Choose smoothing, aggregation, confidence, and filtering settings.
Customise the Plotly chart and threshold line.
Calculate, compare algorithms, and export the report.

Example data

Step	Algorithm	Reward	Optimal reward	Run
1	Epsilon-Greedy	0.55	0.90	1
2	Epsilon-Greedy	0.72	0.92	1
1	UCB	0.61	0.90	1
2	UCB	0.75	0.92	1

Interpretation guide

Realised regret uses observed rewards. Expected regret uses expected_reward when available. Pseudo-regret also compares expected values against the comparator.

A flatter cumulative curve is generally preferable. Confidence bands reveal uncertainty across repeated runs. Wide bands suggest unstable algorithm performance.

Negative regret can indicate noisy rewards. It may also reveal an incorrect comparator. Review suspicious rows before clamping values.

Frequently asked questions

What is a regret curve?

It plots decision loss against an optimal or chosen baseline over time.

Should cumulative regret always increase?

It usually increases when regret is nonnegative. Negative values can reduce it.

What does sublinear regret mean?

Average regret approaches zero as more decisions are made.

Why use repeated runs?

Repeated runs estimate variability and produce meaningful confidence bands.

What is pseudo-regret?

It compares expected rewards instead of only realised random outcomes.

When should I use discounted regret?

Use it when recent decisions matter more than older decisions.

Can this compare multiple algorithms?

Yes. Add algorithm names and matching run identifiers to rows.

Why can regret become negative?

An observed reward may exceed the stated optimal or baseline reward.

Which algorithm is best?

The calculator highlights the algorithm with lowest aggregated cumulative regret.