Game setup
Configure the bandit environment
0.00
0.010
Decision strategy
Choose and tune an algorithm
0.15
0.995
0.02
2.0
0.70
0.10
Custom environment
Edit arm names and hidden reward parameters
| Arm | Name | Mean / probability | Variance | Selection cost |
|---|
True values can stay hidden during play. Use “Reveal values” when reviewing a completed run.
Live arena
Play, simulate, and inspect each decision
Manual choices enabled
+1
Decision typeWaiting
Selected arm—
ReasonStart a run to see algorithm explanations.
Performance
Live metrics
Round
0
Current decision
Total reward
0.00
After costs
Average reward
0.000
Per round
Cumulative regret
0.00
Opportunity loss
Optimal choices
0%
Best arm rate
Exploration
0%
Exploration share
Efficiency
0%
Against oracle
Best identified
—
First stable round
Estimate error
0.000
Mean absolute error
Pending feedback
0
Delayed outcomes
Quiz score
0/0
Concept checks
Strategy stability
0%
Recent consistency
Visual analytics
Plotly.js learning dashboard
Step-by-step learning
Follow the decision update
1
Inspect estimates
Compare the current value estimate for every arm.
2
Apply strategy
Use uncertainty, randomness, or estimated value.
3
Select an arm
Classify the action as exploration or exploitation.
4
Generate reward
Sample from the arm’s hidden distribution.
5
Update knowledge
Revise counts, estimates, confidence, and posteriors.
6
Measure regret
Compare the choice with the current oracle arm.
Strategy laboratory
Compare algorithms on identical environments
| Strategy | Mean reward | Mean regret | Optimal rate | 95% reward interval | Rank |
|---|---|---|---|---|---|
| Run a comparison to populate results. | |||||
Challenges
Select a mission
Knowledge check
Bandit strategy quiz
Decision log
Round-by-round history
| Round | Arm | Decision | Reward | Net reward | Estimate | Best estimate | Instant regret | Cumulative regret | Epsilon |
|---|---|---|---|---|---|---|---|---|---|
| No rounds played. | |||||||||
Concept guide
How the game works
Exploration
Try uncertain arms to gather information. Exploration can reveal a better option, but it may reduce immediate reward.
Try uncertain arms to gather information. Exploration can reveal a better option, but it may reduce immediate reward.
Exploitation
Select the arm with the strongest current evidence. Exploitation earns reward now, but early estimates can be misleading.
Select the arm with the strongest current evidence. Exploitation earns reward now, but early estimates can be misleading.
Regret
Regret measures the expected reward lost by not choosing the current best arm. Lower cumulative regret indicates stronger decisions.
Regret measures the expected reward lost by not choosing the current best arm. Lower cumulative regret indicates stronger decisions.
Non-stationarity
Arm values can drift or change suddenly. Constant step-size updates and continued exploration help algorithms adapt.
Arm values can drift or change suddenly. Constant step-size updates and continued exploration help algorithms adapt.