Regular Markov Chain

An square matrix $A$ is called regular if for some integer $n$ all entries of $A^n$ are positive.

Example

The matrix

$$A = \left[ \begin{array}{rr} 0&1\\ 1&0\\ \end{array} \right]$$

is not a regular matrix, because for all positive integer $n$,

$$A^{2n} = \left[ \begin{array}{rr} 1&0\\ 0&1\\ \end{arra... ...\left[ \begin{array}{rr} 0&1\\ 1&0\\ \end{array} \right]$$

The matrix $A =\left[ \begin{array}{rrrrr} .25&.20&.25&.30 \\ .20&.30&.25&.30 \\ .25&.20&.40&.10 \\ .30&.30&.10&.30 \\ \end{array} \right[$

is a regular matrix, because $A^1$ has all positive entries.

It can also be shown that all other eigenvalues of A are less than 1, and algebraic multiplicity of 1 is one.

It can be shown that if $A$ is a regular matrix then $A^n$ approaches to a matrix $Q$ whose columns are all equal to a probability vector $q$ which is called the steady-state vector of the regular Markov chain.

$$\mbox{ if } A \mbox{ regular, then } A^n \rightarrow Q = \lef... ...&&.\\ .&.&&&&.\\ q_k&q_k&.&.&.&q_k\\ \end{array} \right]$$

where $q_{1} + q_{2} + \dots + q_{k} = 1$.

It can be shown that for any probability vector $x^{(0) }$ when $n$ gets large, $A^n x^{(0)}$ approaches to the steady-state vector

$${\bf q } = \left[ \begin{array}{r} q_1\\ q_2\\ \vdots \\ q_k\\ \end{array} \right]$$

.

That is

$$A^n x^{(0)} \longrightarrow q=\left[ \begin{array}{r} q_1\\ q_2\\ .\\ .\\ .\\ q_k\\ \end{array} \right]$$

where $q_{1} + q_{2} + \dots + q_{k} = 1$.

It can also be shown that the steady-state vector q is the only vector such that

$$Aq = q$$

Note that this shows q is an eigenvector of A and $1$ is eigenvalue of A.