Markov chains and transition matrices – steady state, and walks with adjacency matrices
MathematicsProbabilityAges 17–18
Loading…
Sign in to playBuild a state diagram (people moving between regions, market share, weather, a course with absorbing states) and its transition matrix, using the column or row convention. Multiply the state vector step by step with sₙ₊₁ = T·sₙ, watch the graph converge to the steady state, solve Ts = s and look at the powers Tⁿ. The Adjacency matrix screen counts walks of length n between two vertices with Aⁿ and builds the transition matrix of a random walk on the graph.
Lesson: Markov chains: transition matrices, state vectors, steady state and absorbing states; adjacency matrices and walks of length n
What it shows
A Markov chain moves between states in steps, and the next state depends only on the current one. The transition matrix T holds the conditional probabilities of each move: with the column convention each column sums to 1 and sₙ₊₁ = T·sₙ, while the row convention uses P = Tᵀ with rows summing to 1 and sₙ₊₁ = sₙ·P. The steady state solves Ts = s with entries adding up to 1; a regular chain approaches it from any start, and an absorbing state keeps whatever enters it. A second screen counts walks in a graph with powers of its adjacency matrix. The scenarios are simplified models with illustrative numbers.
How to use
Pick a Scenario and a convention. Type probabilities into the matrix or tap a probability on the diagram and drag the slider; the diagonal entry adjusts itself. Set s₀, then press Step or Play and watch the table and graph. Tick Show matrix powers to see Tⁿ. On Adjacency matrix, choose From, To and Length n, tap two vertices to add or remove an edge, and press Play walks.
Parameters you can change
- Screen Markov chain, Adjacency matrix and walks
- Scenario Weather (2 states), Market share of three brands, Population of three regions, Course (absorbing states)
- Matrix convention Column (s is a column vector, sₙ₊₁ = T·sₙ), Row (s is a row vector, sₙ₊₁ = sₙ·P)
- Steps on the graph 5–50 steps
- Show powers of the transition matrix
- Graph (Adjacency matrix screen) Kite (4 vertices), Square – 4-cycle, House (5 vertices), Directed graph (4 vertices)
- Walk length n 1–8 edges
Questions to explore
- Does the long-run population of each region depend on how many people start in each region?
- In the course scenario, what fraction of new students eventually pass, and how could you raise it?
- Why does a random walk on the square graph never settle down, while the kite graph does?
