ne">\(P_{ij}\)</span> denote the one-step transition probability. That is, <span class="math display">\[ P_{ij} = P(X_{t+1} = j | X_{t} = i)\]</span></p>
<p>In what follows, we will assume that the transition probabilities do not depend on time <span class="math inline">\(t\)</span>. These are called <em>time homogenous</em> Markov chains.</p>
<p>The general idea of simulating discrete Markov chains can be illustrated through a simple example with 2 states. Assume our state space is <span class="math inline