State-Transition Diagram Calculator

Analysis Results

Results update after calculation or simulation.

States

Transitions

Probability status

Not checked

Initial state

—

Terminal states

Reachable states

Expected immediate reward

Steady-state status

—

Validation and warnings

Load data and calculate the diagram.

Key findings

No analysis has been completed.

Project and Data Controls

Load examples, paste data, import files, or restore saved browser sessions.

Project name

Example dataset

Import CSV or JSON

Saved browser session

Paste transitions using Source, Destination, Action, Probability, Reward columns

State Definitions

Configure initial, terminal, grouped, valued, colored, shaped, and manually positioned states.

#	Name	Description	Initial	Terminal	Group	Value	Color	Shape	X	Y	Actions

Transition Definitions

Add action labels, probabilities, rewards, costs, counts, conditions, loops, and bidirectional links.

#	Source	Destination	Action	Probability	Reward	Cost	Count	Label	Condition	Bidirectional	Enabled	Actions

Calculation, Probability, and Reinforcement-Learning Options

Choose validation behavior, Markov steps, policies, rewards, and simulation settings.

Probability handling

Probability display

Decimal precision

Transition steps (n)

Discount factor (γ)

Policy mode

Selected policy action

Reward calculation

Simulation start state

Maximum episode steps

Simulation episodes

Random seed

Stop episodes at terminal states

Include disabled transitions

Add implicit terminal self-loops

Animate single episode

Interactive State-Transition Diagram

Drag states, change layouts, filter edges, highlight paths, and export the finished graph.

Layout

Edge style

Edge labels

Node labels

Node size

Label font size

Minimum probability

Minimum reward

Maximum reward

Action filter

State search

Edge color

Hide self-loops

Hide isolated states

Terminal paths only

Lock node positions

Show conditions

Show counts

Path start

Path destination

Select start and destination states to highlight a path.

Matrices and Detailed Analysis

Inspect transition, reward, multi-step, stationary, policy, absorption, and visitation outputs.

No matrix available.

No steady-state calculation available.

No absorption calculation available.

No policy calculation available.

No simulation has been run.

Formula Used

The calculator applies standard Markov-chain and reinforcement-learning relationships.

Transition matrix:
P_ij = Pr(X_t+1 = j | X_t = i)

n-step distribution:
π⁽ⁿ⁾ = π⁽⁰⁾Pⁿ

Stationary distribution:
π = πP, with Σπ_i = 1

Bellman expectation:
V(s) = Σ_aπ(a|s)Σ_s′P(s′|s,a)[R + γV(s′)]

Fundamental matrix:
N = (I − Q)⁻¹

Expected absorption steps:
t = N1

How to Use

Add state names and mark the initial or terminal states.
Add directed transitions with actions, probabilities, and rewards.
Choose probability, policy, layout, and simulation options.
Select Calculate and draw to validate and analyze the model.
Inspect matrices, paths, steady-state values, and absorption results.
Run episodes, then export the diagram or calculated tables.

Example Data

Source	Destination	Action	Probability	Reward
Start	State A	Move	0.70	2
Start	State B	Wait	0.30	-1
State A	End	Finish	1.00	10
State B	State A	Retry	0.60	1
State B	End	Quit	0.40	-5

Frequently Asked Questions

What is a state-transition diagram?

It shows system states as nodes. Directed arrows show possible transitions. Labels can display actions, probabilities, rewards, or conditions.

Can this calculator model Markov chains?

Yes. It creates transition matrices and multi-step probabilities. It also estimates stationary and absorbing-chain behavior.

Must outgoing probabilities total one?

Strict mode requires totals of one. Normalize mode adjusts positive values automatically. Allow mode keeps incomplete probability totals.

How are terminal states handled?

Terminal states can stop simulations. Optional self-loops make them absorbing. Absorption calculations use the resulting canonical matrix.

Can I model reinforcement-learning rewards?

Yes. Add rewards, costs, actions, and discounting. The calculator estimates state values using Bellman iteration.

What does steady state mean?

It is a long-run probability distribution. Applying the transition matrix leaves it unchanged. Some chains have no unique stationary result.

Can I import transition data?

Yes. Import CSV or JSON files. You can also paste comma-separated transition rows.

How are multiple transitions combined?

Transitions between identical state pairs are aggregated. Their probabilities are added. Reward values use probability-weighted averages.

Can I export the diagram?

Yes. Export PNG, SVG, PDF, CSV, or JSON. Browser sessions can also be saved locally.

Current Configuration Preview

Configuration preview will appear here.