">\(\{1,2\}\)</span> and the transition matrix is:</p>
<p><span class="math display">\[P = \begin{bmatrix}
    0.7 &amp; 0.3 \\
    0.4 &amp; 0.6
\end{bmatrix}\]</span></p>
<p>We denote the <span class="math inline">\((i,j)\)</span>-th entry of the matrix <span class="math inline">\(P\)</span> as <span class="math inline">\(P_{ij}\)</span>.</p>
<p>Now assume that our Markov chain starts in state 1 so that <span class="math inline">\(X_0 = 1\)</span>. Since we are starting in state 1, our transition probabil