the inference and learning stage. Factorized components can be imposed to a mixture model without loss of generality. Any statistical dependence between variables can be still captured, at the cost of a possibly larger number of components in the mixture to achieve the required accuracy in the approximation.

The simplicity of the "building block" structure is entirely compensated by an important saving in computation time. High-dimensional integrals are analytically computed from univariate integrals and matrix inversions are avoided in the learning stage. Additionally, high-dimensional domains can be easily modeled using a small number of parameters in each mixture component.

From the viewpoint of Computational Learning Theory (Vapnik 1995), models with a small number of adjustable parameters (actually, with low "expressive power") have favorable consequences for generalization.

Mixtures of factorized components are also used in Latent Class Analysis (DeSoete 1993), a well-known unsupervised classification technique. It is assumed that the statistical dependences between attributes can be fully explained by a hidden variable specifying the "latent class" of each example. This method is similar to the Gaussian decomposition clustering algorithm mentioned in Section 1, constrained to component-conditional attribute independence. However, our goal is not unsupervised classification but obtaining an accurate and mathematically convenient expression for the joint density of the variables, required to derive the desired estimators. The meaning of the components is irrelevant, as long as the whole mixture is a good approximation to the joint density.

More expressive architectures, which combine mixture models with local dimensionality reduction, have been also considered: Mixtures of Linear Experts (Jordan & Jacobs 1994), Mixtures of Principal Component Analyzers (Sung & Poggio 1998) or Mixtures of Factor Analyzers (Ghahramani & Hinton 1996, Hinton, Dayan, & Revow 1997). Unfortunately, the general kind of inference and learning from uncertain data considered in this work cannot be directly incorporated into these architectures with the computational advantages demonstrated by the MFGN model.

The restriction to factorized components may produce undesirable artifacts in the approximations of certain domains learned from small training samples. Nevertheless, this problem always occurs to any approximator when the structure of the building block does not match the "shape" of the target function. In this case, many terms (or components, units, etc.) are required for a good approximation and the associated parameters can be correctly adjusted only from a large training sample. However, note that the complexity of the model should not be measured uniquely in terms of the number of mixture components. The number of adjustable parameters is probably a better measure of complexity. For instance, full covariance models show a quadratic growth of the number of free parameters with respect to the dimension of the attribute vector. For factorized components the growth is linear, so the amount of training data need not be unreasonably high even if the number of mixture components is large.

In real applications, the nature of the target function is unknown, so little can be said *a priori* about the best building block structure to be used by a universal approximator. We have chosen a very simple component structure to make inference and learning feasible from uncertain information. Section 5.4 provides experimental evidence that in realistic problems the proposed model is not inferior to other popular approaches.

## 5.3 Qualitative Comparison With Alternative Approaches

Instead of the proposed methodology, based on mixture models and the EM algorithm, other alternative nonparametric density approximation methods could also be used (either for the joint density or for specific conditional densities). For instance, the nearest neighbor rule locally approximates the target density using a certain number of training samples near to the point of interest. Symbolic