
$$(\sigma_{i}^{j})^{2}\leftarrow\frac{1}{M P_{i}}\sum_{k}\sum_{r}\left[q_{i,r}[(\mathbf{v}_{i,r}^{j})^{2}+(\lambda_{i,r}^{j})^{2}]\right]^{(k)}-(\mu_{i}^{j})^{2}$$

For symbolic variables under representation (8) we may use:

$$\beta_{i,r}^{j\,(k)}=\sum_{\omega}\mathrm{P}\{s_{r}^{j}=\omega\}^{(k)}t_{i,\omega}^{j}$$
, = ∑P{ = } (26)'
$$t_{i,\omega}^{j}\leftarrow\frac{1}{MP_{i}}\sum_{k}\sum_{r}\left[q_{i,r}\ \mathrm{P}\{s_{r}^{j}=\omega\}\ t_{i,\omega}^{j}\ /\ \beta_{i,r}^{j}\ \right]^{(k)}\tag{27}$$
$$(27)$$
$$(26)^{\circ}$$

Consider the particular case in which the attributes in the training examples are contaminated with unbiased Gaussian noise. The likelihood of the uncertain observations is modeled by 1component mixtures: p z s k k j j k j k j k
( | ) ( , , )
( ) ( ) ( ) ( ) ( )
s z = Π T ε , where ( )
( )
ε j k 2is the variance of the measurement process over z j ( k ) which obtains the observed value s j ( k ). This can be also expressed as a confidence interval z s j ( k ) j ( k ) j ( k )
≅ ± 2ε . In this case, the basic EM algorithm (23)
can be easily modified to take into account the effect of the uncertainties ε j ( k ). In the E step, compute qi
( k ) using the following deviations:

$$\sigma_{i}^{j}\leftarrow\sqrt{(\sigma_{i}^{j})^{2}+(\epsilon^{j\,(k)})^{2}}$$

and, in the M step, apply the substitution: where

$$z^{j\,(k)}\leftarrow\gamma\,s^{j\,(k)}+(1-\gamma)\,\mu_{i}^{j}$$  $$(z^{j(k)})^{2}\leftarrow\left[\gamma s^{j(k)}+(1-\gamma)\mu_{i}^{j}\right]^{2}+\gamma\left[\epsilon^{j(k)}\right]^{2}$$
$$\gamma=\frac{\left(\sigma_{i}^{j}\right)^{2}}{\left(\sigma_{i}^{j}\right)^{2}+\left(\epsilon^{j(k)}\right)^{2}}$$

measures the relative importance of the observed s j ( k )for computing the new µi jand σi j.

The previous situation illustrates how missing values must be processed in the learning stage.

If z j ( k )is exact then ε j ( k )= 0 and γ = 1, so the original algorithm (23) is not changed. In the other extreme, if z j ( k )is missing, which can be modeled by ε j ( k )≡ ∞, we get γ = 0 and therefore the observation s j ( k ) does not contribute to the new parameters at all. The correct procedure to deal with missing values *in the MFGN framework* is simply omitting them in the empirical averages.

Note that this fact arises from the factorized structure of the mixture components, providing conditionally independent attributes. Alternative learning methods require a careful management of missing data to avoid biased estimators (Ghahramani & Jordan 1994).

## 4.3 Evaluation Of The Extended Em Algorithm

We have studied the improvement of the parameter estimations when the uncertainty of the observations, modeled by likelihood functions, is explicitly taken into account. The proposed Extended