attributes are directly estimated by a voting scheme and continuous attributes can be also estimated by averaging the observed values of training instances which are near, in the subspace of observed attributes, to the point of interest. However, for small sample sizes, the above estimators are not smooth and show strong sensitivity to random fluctuations in the training set, which penalizes the estimation cost. For large sample size, the time required to find the nearest neighbors becomes very long. As an example, consider the regression problem in Example 10, Section 3.5. Fig. 11.b shows the MFGN solution with 4 components and MSE=0.381. Fig. 21.a shows the regression line obtained by 5-nearest-neighbors average, with a higher MSE=0.522.

Parzen windows and similar kernel approximation methods are used to smooth the results of the simple nearest neighbors rule (Duda & Hart 1973, Izenman 1991). They are actually mixtures of simple conventional densities located at the training samples. In principle, the properties of the MFGN framework could be adapted to that kind of approximation (Ruiz et al. 1998). Learning becomes trivial, but strong run time computation effort is required since a "concise" model of the domain is not extracted from the training set. This kind of rote learning has also negative consequences on generalization according to the Occam Razor Principle (Li & Vitànyi 1993). An adequately cross-validated mixture model with a small number of components in relation to the training sample size reasonably guarantees that probably the true attribute dependencies are correctly captured.

![0_image_0.png](0_image_0.png)

The nature of the solutions obtained by Backpropagation Multilayer Perceptrons (Rumelhart et al. 1986) in pattern classification is also illustrative. In general, each decision region can be geometrically expressed as the union of intersections of several half–spaces defined by the units in the first hidden layer. However, backprop networks often require very long learning times, many adjustable parameters and, what is worse, apparently simple distributions of patterns are hard to learn. For instance, the solution to the circle-ring classification problem in Fig. 21.b, obtained by a network with 6 hidden units requires hundreds of standard backprop epochs. The decision regions are not very satisfactory, even though the network has extra flexibility for this task (3 hidden units suffice to separate the training examples). Better solutions exist using all the resources in the network architecture, but backprop learning does not find them. In contrast, the solution obtained by the MFGN approach using 7 components (Fig. 21.c) requires a learning time orders of magnitude shorter than backprop optimization. All the components in the mixture contribute to synthesize reliable decision regions and acceptable solutions can be also obtained with a smaller number of components.