Finding the Stationary Distribution for a Transition Matrix

  By a stationary distribution for a random process with transition matrix P we mean a system state vector π which remains unchanged after operating on it with P, i.e., Pπ=π. Given P, how do can we determine π? A simple method is to compare the definition of the stationary state with the definition of a eigenvector and deduce a relation connecting the state vector with the eigenvector.

Setting the eigenvalue μ=1 makes the two equations nearly identical. So if we can find an eigenvector of P which has an eigenvalue equal to 1 then we can find the direction of π. To find the magnitude in the given direction we use the fact that the sum of the components of π is one to determine the multiplier for e. We can illustrate how this works by determining the stationary state for a simple random walk problem. In the state diagram below q=1-p:

The determination of the stationary state proceeds as follows:

There is only one stationary state for the problem above and we find that each state has equally probable. One can see that more that one stationary state is possible for a given transition matrix by considering a simple cascade problem which starts in state 0 and can end up in either states 1 or 2.

Looking at the eigenvectors for the transition matrix we see that there are two with eigenvalues equal to 1.

Here each column of π is a stationary state corresponding to final states 1 and 2.