World canvas
Learning dashboard
Recent results
| Episode | Reward | Steps | Result | Epsilon |
|---|---|---|---|---|
| No episodes yet. | ||||
World setup
Learning controls
Environment rules
Move the explorer
Use arrow keys or WASD. Click any cell to inspect or edit it.
Paint cell types
Selected cell
Formula and decision explanation
The explorer has not moved yet. Start an episode to see each learning update explained.
Blue arrows show the current preferred action. Cell shading represents estimated state value.
Save, export, and report
Explore, train, compare, and improve
- Choose a preset or paint your own map.
- Select an algorithm and tune its learning parameters.
- Move manually, train one step, or run many episodes.
- Inspect policy arrows, Q-values, rewards, routes, and convergence.
- Test the learned policy and export your results.
Grid world learning questions
What is a policy?
A policy selects an action for each state. The arrows display the explorer's current policy.
Why does the agent explore?
Exploration discovers routes that greedy choices might miss. Epsilon controls how often random actions occur.
What does the discount factor do?
Gamma controls how strongly future rewards affect current decisions. Higher values encourage longer-term planning.
How is Q-learning different from SARSA?
Q-learning learns toward the best next action. SARSA learns from the next action actually selected.
What does convergence mean?
Convergence means preferred actions stop changing significantly. The stability meter estimates that behaviour.
Can the world be stochastic?
Yes. Enable stochastic movement to make intended actions occasionally drift sideways.
Why can a longer route be better?
A longer path may collect useful rewards or avoid severe penalties. Total discounted return determines value.
What are terminal cells?
Goals and traps end an episode. Their rewards strongly shape learned behaviour.
Can I save custom maps?
Yes. Custom maps and full learning sessions use browser storage.