Results
Enter episode rewards or load sample data. Calculated results appear here.
Interactive reward curve
Training interpretation
Series comparison
Calculated episode data
Formula used
Average reward: R̄ = (1 / N) × ΣRi Cumulative reward: Cn = ΣRi Rolling mean: MAt = (1 / w) × ΣRi Exponential moving average: EMAt = αRt + (1 − α)EMAt−1 Reward improvement: Rfinal − Rinitial Percentage improvement: ((Rfinal − Rinitial) / |Rinitial|) × 100 Trend slope: Σ((xi − x̄)(yi − ȳ)) / Σ((xi − x̄)²) Coefficient of variation: standard deviation / |mean reward|
How to use
Enter one reward value per line. Optional episode numbers may precede rewards. Separate both values using commas.
Add series for agent comparisons. Choose smoothing and threshold settings. Then calculate the complete reward analysis.
Review charts, statistics, and warnings. Export results when needed. Keep original experiment logs for verification.
Example data table
| Episode | PPO training reward | PPO evaluation reward |
|---|---|---|
| 1 | -42 | -35 |
| 2 | -18 | -10 |
| 3 | 4 | 12 |
| 4 | 28 | 35 |
| 5 | 51 | 62 |
Frequently asked questions
What does an episode reward curve show?
It shows how total reward changes across reinforcement-learning episodes. Rising curves usually indicate learning progress.
Why should rewards be smoothed?
Episode rewards are often noisy. Smoothing reveals longer-term learning trends without removing the raw observations.
Which smoothing method should I choose?
Moving averages are easy to interpret. EMA reacts faster, while median smoothing resists isolated spikes.
What is a solved threshold?
It is a reward level used to define successful training. Many environments require consecutive successful episodes.
How is convergence estimated?
The calculator checks smoothed rewards, slope, stability, and consecutive threshold performance. This remains an estimate.
Can negative rewards be analyzed?
Yes. Negative, zero, and positive rewards are accepted. Logarithmic scaling requires strictly positive plotted values.
Can multiple agents be compared?
Yes. Add several series and identify each algorithm, environment, run, and training designation.
What indicates unstable training?
Large reward volatility, repeated collapses, high variance, or worsening late performance may indicate instability.
What is reward drawdown?
Maximum drawdown measures the largest decline from a previous reward peak during the selected episodes.
How does CSV import work?
Use long format with episode, series, and reward columns. Wide files may place each series in separate columns.
Can calculated data be exported?
Yes. Export CSV, PDF, PNG, or SVG files. You may also copy or print results.