Clearly, each Siis a measurable partition of X. Since X is Hausdorff, for each pair **x, x**′ of distinct points from X, there is a pair of disjoint open sets A and A′such that x ∈ A and x
′ ∈ A′. Since S
is dense, there is a pair **s, s**′from S with s ∈ A and s
′ ∈ A′. Also, N contains neighbourhoods Ns and Ns
′ with Ns ⊆ A and Ns
′ ⊆ A′. So Ns and Ns
′ are disjoint. Thus, for sufficiently large i, x and x
′fall in distinct elements of the partition Si. Since this is true for any pair **x, x**′, it follows that

$$\bigcap_{i=1}^{\infty}\{S\in{\mathcal{S}}_{i}:x\in S\}\subseteq\{x\}.$$

The reverse inclusion is trivial. The measurability of all singletons {x} follows from the measurability of Sx := Si{S ∈ Si: S ∩ {x} = φ} and the fact that {x} = X − Sx.

We shall use Lemma 7 together with the following result to show that we can approximate expectations of certain random variables using a single sample path of the Markov chain.

Lemma 8. Let(X,B) *be a measurable space satisfying the conditions of Lemma 7, and let* S1, S2**, . . .**
be a suitable sequence of partitions as in that lemma. Let µ be a probability measure defined on this space. Let f be an absolutely integrable function on X. For an event S*, define*

$$f(S)={\frac{\int_{S}f\,d\mu}{\mu(S)}}.$$

For each x ∈ X and k = 1, 2**, . . .**, let Sk(x) be the unique element of Sk containing x. Then for almost all x in X,

$$\operatorname*{lim}_{k\to\infty}f(S_{k}(x))=f(x).$$

Proof. Clearly, the signed finite measure φ defined by

$$\phi(E)=\int_{E}fd\mu\tag{42}$$

is absolutely continuous with respect to µ, and Equation (42) defines f as the Radon-Nikodym derivative of φ with respect to µ. This derivative can also be defined as

$$\frac{d\phi}{d\mu}(x)=\operatorname*{lim}_{k\rightarrow\infty}\frac{\phi(S_{k}(x))}{\mu(S_{k}(x))}.$$

See, for example, (Shilov & Gurevich, 1966, Section 10.2). By the Radon-Nikodym Theorem (Dudley, 1989, Theorem 5.5.4, p. 134), these two expressions are equal a.e. (µ).

Proof. (Theorem 6.) From the definitions,

$$\nabla_{\!\beta}\eta=\pi^{\prime}\nabla P J_{\beta}$$
$$=\sum_{i=1}^{n}\sum_{j=1}^{n}\pi(i)\nabla p_{ij}(\theta)J_{\beta}(j).\tag{43}$$

For every y, µ is absolutely continuous with respect to the reference measure λ, hence for any i and j we can write

$$p_{i j}(\theta)=\int_{\mathcal{Y}}\int_{\mathcal{U}}p_{i j}(u)\,\frac{d\mu(\theta,y)}{d\lambda}(u)\,d\lambda(u)\,d\nu(i)(y).$$
$$345$$
