
$$\operatorname{E}_{\mathbf{z}|C_{i}}\left\{g(\mathbf{z})\,|\,C_{i}\right\}=\operatorname{E}_{S}\left\{\sum_{r}\alpha_{i,r}\int_{Z}g(\mathbf{z})\prod_{j}\psi_{i,r}^{j}(z^{j})d\mathbf{z}\right\}/P_{i}$$

In the MFGN framework the contributions Γi,r r Ci r
(s ) P{ ,s |S} to the empirical expected values required by the Extended EM algorithm can be obtained again without numeric integration.

We only need to consider the case g(z) = z jto compute the means µi jand probabilities j i t,ω
, and g(z) = (z j)
2for the deviations σi j. From (12) we already know an explicit expression for the parameters of ψi r ν λ j j j i r j i r j z z , , ,
( ) = T( , , ). Hence:

$$\int_{z}z^{j}\Psi_{i,r}(z)\,d z=\int_{z}z^{j}\Psi_{i,r}^{j}(z^{j})d z^{j}=\Psi_{i,r}^{j}$$
$$\int_{z}(z^{j})^{2}\,\Psi_{i,r}(z)\,d z=(\nu_{i,r}^{j})^{2}+(\lambda_{i,r}^{j})^{2}$$

In conclusion, the steps of the Extended EM algorithm are as follows:
(E) *Expectation step.* Compute all the elementary likelihoods of the training set:

$$\beta_{i,r}^{j\,(k)}={\mathcal{N}}\Big(s_{r}^{j},\mu_{i}^{j},{\sqrt{(\sigma_{i}^{j})^{2}+(\epsilon_{r}^{j})^{2}}}\Big)^{(k)}$$
2 2(26)
Obtain the likelihood of each conjunction sr
( k ) of example S

$${\mathrm{\boldmath~\ale~}}S^{(k)}{\mathrm{\scriptsize~in~component~}}C_{i}.$$
$$(26)$$
$$\mathrm{m}\ \mathbf{s}_{r}^{+}$$
$$\beta_{i,r}^{(k)}=\prod_{i}\beta_{i,r}^{j\,(k)}$$

Obtain the total likelihood of example S
( k ):

$$\beta^{(k)}\equiv p(S^{(k)})=\sum_{i}\sum_{r}P_{i}\ \pi_{r}^{(k)}\ \beta_{i,r}^{(k)}$$

Compute the probabilities qi r C
k i r k k
,
( ) ( ) ( )
≡ P{ ,s |S } that the r-th component of the k-th example has been generated by the i-th component of the mixture:

$$q_{i,r}^{(k)}\leftarrow\alpha_{i,r}^{(k)}=P_{i}\ \pi_{r}^{(k)}\ \beta_{i,r}^{(k)}\ /\ \beta^{(k)}$$

(M) *Maximization step*. Update the parameters of each component Ci using all the components sr
( k ) of all the examples weighted with their probabilities qi r k
,
( ). First, the prior probabilities of each component:

$$P_{i}\leftarrow\frac{1}{M}\sum_{k}\sum_{r}q_{i,r}^{(k)}$$

Then the mean value and standard deviation in each component:

$$\mu_{i}^{j}\leftarrow\frac{1}{M P_{i}}\sum_{k}\sum_{r}\Bigl[q_{i,r}\ \mathbf{v}_{i,r}^{j}\Bigr]^{(k)}$$
