information:filtering
Time-homogeneous Markov chain with a finite state space

If the Markov chain is time-homogeneous, then the transition matrix P is the same after each step, so the k -step transition probability can be computed as the k -th power of the transition matrix, P.

If the Markov chain is irreducible and aperiodic, then there is a unique stationary distribution π. Additionally, in this case P converges to a rank-one matrix in which each row is the stationary distribution π:

where 1 is the column vector with all entries equal to 1. This is stated by the Perron–Frobenius theorem. If, by whatever means, lim k → ∞ P k {\textstyle \lim _{k\to \infty }\mathbf {P} ^{k}} is found, then the stationary distribution of the Markov chain in question can be easily determined for any starting distribution, as will be explained below.

For some stochastic matrices P, the limit lim k → ∞ P k {\textstyle \lim _{k\to \infty }\mathbf {P} ^{k}} does not exist while the stationary distribution does, as shown by this example:

(This example illustrates a periodic Markov chain.)

Because there are a number of different special cases to consider, the process of finding this limit if it exists can be a lengthy task. However, there are many techniques that can assist in finding this limit. Let P be an n × n matrix, and define Q = lim k → ∞ P k. {\textstyle \mathbf {Q} =\lim _{k\to \infty }\mathbf {P} ^{k}.}

It is always true that

Subtracting Q from both sides and factoring then yields

where I is the identity matrix of size n, and 0 is the zero matrix of size n × n. Multiplying together stochastic matrices always yields another stochastic matrix, so Q must be a stochastic matrix (see the definition above). It is sometimes sufficient to use the matrix equation above and the fact that Q is a stochastic matrix to solve for Q. Including the fact that the sum of each the rows in P is 1, there are n+1 equations for determining n unknowns, so it is computationally easier if on the one hand one selects one row in Q and substitutes each of its elements by one, and on the other one substitutes the corresponding element (the one in the same column) in the vector 0, and next left-multiplies this latter vector by the inverse of transformed former matrix to find Q.

Here is one method for doing so: first, define the function f (A) to return the matrix A with its right-most column replaced with all 1's. If exists then

One thing to notice is that if P has an element P on its main diagonal that is equal to 1 and the i th row or column is otherwise filled with 0's, then that row or column will remain unchanged in all of the subsequent powers P. Hence, the i th row or column of Q will have the 1 and the 0's in the same positions as in P.