Transition probability
\(p_{ij} = P(X_{n+1}=j \mid X_n=i)\). Row \(i\) describes the next-state probabilities from state \(i\).
Markov Chain Calculator & Simulator
Inputs
Start with an example
Build a square transition matrix. Rows are current states; columns are next states.
25%, or fractions such as 1/4.
Import, export, exact calculations, and simulations are client-side. Your matrix is not uploaded by this page.
Calculator Results
Load an example or enter states, then calculate.
Tips: row sums should be 1. Use Normalize or presets to create a valid transition matrix.
Empirical vs exact or stationary distribution
Short simulations vary because they sample random paths. Increasing steps or runs usually moves empirical visit frequencies toward the exact n-step or long-run stationary distribution when the chain has a stable long-run behavior.
State graph
Nodes are states.Arrows show nonzero transition probabilities.The active node follows the latest random walk.
Advertisement
Markov chain formulas used by this calculator
Row-stochastic matrix
\(\sum_j p_{ij}=1\) for every row, with \(p_{ij}\ge 0\). The tool flags rows that do not sum to 1.
n-step distribution
\(\pi_n = \pi_0 P^n\). This gives exact probabilities after \(n\) transitions from your chosen initial distribution.
Stationary distribution
\(\pi P = \pi\) and \(\sum_i \pi_i=1\). When unique, it is the expected long-run proportion of time in each state.
Worked examples and interpretation
A Markov chain models movement between states where the next state depends only on the current state. This page combines calculator-style exact results with a random walk simulator: exact tables answer probability questions, while sampled paths show one possible sequence of states.
Weather example
In the default weather matrix, the Sunny row 0.7, 0.2, 0.1 means a sunny day is followed by Sunny with
probability 0.7, Cloudy with probability 0.2, and Rainy with probability 0.1. \(P^n\) answers questions such as
“starting from Sunny, what is the probability of Rainy after 10 steps?”
Absorbing-state example
An absorbing state has a 1 on its diagonal and zeros elsewhere in its row. Once the random walk enters that state, it stays there. The diagnostics panel identifies absorbing states and whether parts of the chain are unreachable.
Common mistakes
- Columns instead of rows: this calculator uses row vectors, so rows must sum to 1.
- Confusing simulation with exact probability: a short random walk can look noisy even when the exact distribution is stable.
- Assuming every chain has one steady state: reducible, periodic, or absorbing chains can have long-run behavior that depends on the start.
Methodology and privacy
The simulator samples each next state from the current row of \(P\). Exact \(P^n\) and \(\pi_0P^n\) are computed by matrix multiplication. The stationary result is found by fixed-point iteration from a uniform distribution and checked against \(\pi P = \pi\); diagnostics note when uniqueness may not be expected.
Import, export, graph drawing, random walks, matrix powers, and diagnostics run in browser JavaScript. The page does not send your matrix or paths to a server.
Markov chain FAQ
How do I find the steady-state distribution?
Enter a valid transition matrix and run the calculator. The stationary panel reports \(\pi\) where \(\pi P = \pi\) and the entries sum to 1.
What does \(P^n\) mean?
\(P^n\) is the n-step transition matrix. Entry \(i,j\) is the probability of being in state \(j\) after \(n\) transitions when starting in state \(i\).
Why must rows sum to 1?
Each row lists all possible next states from one current state, so the probabilities must be nonnegative and total 1.
What is the difference between simulation and exact probability?
Simulation samples random paths, so results vary. Exact probability uses matrix multiplication and is deterministic for the same inputs.
What is an absorbing state?
An absorbing state has probability 1 of staying in itself. Once reached, the random walk cannot leave that state.
How many steps should I run?
Use small step counts for demonstrations and larger steps or multiple runs to compare empirical frequencies with exact or stationary probabilities.
Does this handle continuous-time Markov chains?
No. This calculator is for finite, discrete-time Markov chains with a row-stochastic transition matrix.