information:highlighting, linguistic:paraphrasing, quality:correction
Embedded Markov chain

One method of finding the stationary probability distribution, π, of an ergodic continuous-time Markov chain, Q, is by first finding its embedded Markov chain (EMC). Strictly speaking, the EMC is a regular discrete-time Markov chain, sometimes referred to as a jump process. Each element of the one-step transition probability matrix of the EMC, S, is denoted by s, and represents the conditional probability of transitioning from state i into state j. These conditional probabilities may be found by

From this, S may be written as

where I is the identity matrix and diag(Q) is the diagonal matrix formed by selecting the main diagonal from the matrix Q and setting all other elements to zero.

To find the stationary probability distribution vector, we must next find φ {\displaystyle \varphi } such that

with φ {\displaystyle \varphi } being a row vector, such that all elements in φ {\displaystyle \varphi } are greater than 0 and ‖ φ ‖ 1 {\displaystyle \|\varphi \|_{1}} = 1. From this, π may be found as

(S may be periodic, even if Q is not. Once π is found, it must be normalized to a unit vector.)

Another discrete-time process that may be derived from a continuous-time Markov chain is a δ-skeleton — the (discrete-time) Markov chain formed by observing X (t) at intervals of δ units of time. The random variables X (0), X (δ), X (2δ),... give the sequence of states visited by the δ-skeleton.