
| Case                                                               | P1   | µ1 x   | µ1 y   | σ1 x   | σ1 y   | t white w   | x    | µ2 y   | σ2   |      |      |
|--------------------------------------------------------------------|------|--------|--------|--------|--------|-------------|------|--------|------|------|------|
| 1,                                                                 | µ2   |        |        |        |        |             |      |        |      |      |      |
| true                                                               | .5   | 0      | 0      | 2      | 1      | 1           | 2    | 2      | 1    | 2    | 1    |
| 0                                                                  | .48  | -.04   | .03    | 2.11   | 1.00   | 1.00        | 2.09 | 1.83   | 1.00 | 2.08 | 1.00 |
| 20%                                                                | .48  | .16    | -.03   | 1.91   | 1.01   | 0.96        | 2.31 | 2.09   | .88  | 2.10 | 1.00 |
| 40%                                                                | .49  | .14    | -.16   | 1.78   | .99    | 0.95        | 2.39 | 2.29   | .94  | 2.14 | 1.00 |
| 60%                                                                | .45  | .02    | -.29   | 2.50   | .78    | 1.00        | 1.86 | 2.01   | 1.03 | 1.77 | 1.00 |
| 80%                                                                | .49  | -.11   | 1.69   | 2.21   | 1.73   | .50         | 1.91 | 0.31   | 1.14 | 0.68 | 0.47 |
| 1                                                                  | .48  | .06    | .90    | 1.88   | 1.73   | 1.00        | 1.88 | 2.98   | .96  | 2.40 | 1.00 |
| 2                                                                  | .48  | -.04   | .05    | 1.90   | .92    | 1.00        | 1.98 | 1.97   | 1.01 | 1.90 | 1.00 |
| 3                                                                  | .47  | .02    | .09    | 1.60   | 1.02   | .87         | 2.00 | 1.78   | 1.14 | 2.07 | 0.85 |
| 4                                                                  | .49  | .27    | -.08   | 1.97   | .90    | .70         | 2.10 | 2.06   | 1.01 | 1.94 | 0.71 |
| 5                                                                  | .43  | -.04   | -.18   | 2.40   | 1.48   | .82         | 1.85 | 1.52   | 1.52 | 2.33 | .80  |
| 6                                                                  | .54  | -.07   | -.02   | 1.97   | 1.09   | .56         | 1.93 | 2.11   | .85  | 1.69 | .62  |
| 7                                                                  | .46  | .15    | -.16   | 2.73   | 2.53   | .31         | 1.94 | 1.62   | 2.47 | 2.90 | .29  |
| 8                                                                  | .79  | .87    | .08    | 2.09   | 1.52   | .51         | 1.96 | 3.61   | 0.87 | 1.21 | .54  |
| 9                                                                  | .48  | .32    | -.02   | 1.77   | 1.10   | 1.00        | 1.92 | 0.67   | 0.94 | 1.22 | 1.00 |
| 10                                                                 | .45  | .00    | .03    | 2.20   | 1.01   | 1.00        | 2.13 | 1.55   | 1.04 | 1.77 | 1.00 |
| Table 6. Parameter Estimates from Uncertain Information (see text) |      |        |        |        |        |             |      |        |      |      |      |

training set test set prediction error 0% 0% 2.7%

missing attributes 0% 50% 12.0%

50% 0% 4.0%

50% 50% 18.7%

 In the relatively simple IRIS domain, the performance degradation due to 50% missing attributes is much greater in inference than in learning stage. The Extended EM algorithm is able to correctly recover the overall structure of the domain from the available information.

## 4.4 Comments

Convergence of the EM Algorithm is very fast, requiring no adjustable parameters such as learning rates. The algorithm is robust with respect to the random initial mixture parameters: bad local maxima are not frequent and alternative solutions are usually equally acceptable. All the examples contribute to all the components, which are never wasted by unfortunate initialization. For a fixed number of components, the algorithm progressively increases the likelihood J of the training data until a maximum is reached. When the number of components is incremented the maximum J also increases, until a limit value is obtained that cannot be improved using extra components (Fukunaga 1990). Some simple heuristics can be incorporated to the standard Expectation-Maximization scheme to control the value of certain parameters (e.g., lower bounds can be established for variances) or the quality of the model (e.g., mixture components can be eliminated if their proportions are too small).

In our case, factorized components are specially convenient because matrix inversions are not required and, what is more important, uncertain and missing values can be correctly handled in a