(Non)-homogeneity of Markov chain

by Alvin Lepik   Last Updated September 11, 2019 10:20 AM

Suppose we play a board game, where we roll the dice and step forward corresponding number of steps. Suppose there are $m$ squares on the board and first one to arrive on square $m$ wins.

A knee jerk reaction would be to say that there are $m$ states: $1,\ldots, m$ and $$\mathbb P (X_n = j \mid X_{n-1} = i) =: p_{ij} $$ that is the probability of getting to square $j$ assuming we started from $i$.

But this feels wrong, because if there are, say, $5$ squares and we toss heads or tails to decide whether we move forward one or two squares, then this probability $p_{ij}$ is not independent of $n$. In fact, the game can last at most $4$ turns, because we're always guaranteed to move ahead at least one square. Same thing with the dice roll.

The theory I've encountered restricts itself to the homogeneous case. Can we 'fictitiously' convert a non-homogeneous chain to one that does not depend on time or how would we analyse a non-homogeneous chain?



Related Questions


Markov distribution

Updated April 21, 2017 16:20 PM



Expected value for Markov CHAIN

Updated December 12, 2017 00:20 AM