
## 4. Multiple Kernel Embedding

In the previous section, we derived an algorithm to learn an optimal projection from a kernel space H to R
dsuch that Euclidean distance between embedded points conforms to perceptual similarity.

If, however, the data is heterogeneous in nature, it may not be realistic to assume that a single feature representation can sufficiently capture the inherent structure in the data. For example, if the objects in question are images, it may be natural to encode texture information by one set of features, and color in another, and it is not immediately clear how to reconcile these two disparate sources of information into a single kernel space.

However, by encoding each source of information independently by separate feature spaces H1, H2**, . . .** - equivalently, kernel matrices K1, K2**, . . .** - we can formulate a multiple kernel learning algorithm to optimally combine all feature spaces into a single, unified embedding space. In this section, we will derive a novel, projection-based approach to multiple-kernel learning and extend Algorithm 3 to support heterogeneous data in a principled way.

## 4.1 Unweighted Combination

Let K1, K2**, . . . , K**m be a set of kernel matrices, each with a corresponding feature map φ pand RKHS Hp, for p ∈ 1**, . . . , m**. One natural way to combine the kernels is to look at the product space, which is formed by concatenating the feature maps:

$$\phi(x_{i})=(\phi^{1}(x_{i}),\phi^{2}(x_{i}),\ldots,\phi^{m}(x_{i}))=(\phi^{p}(x_{i}))_{p=1}^{m}.$$

Inner products can be computed in this space by summing across each feature map:

$$\langle\phi(x_{i}),\phi(x_{j})\rangle=\sum_{p=1}^{m}\langle\phi^{p}(x_{i}),\phi^{p}(x_{j})\rangle_{\mathcal{H}^{p}}\,.$$

resulting in the *sum-kernel* - also known as the average kernel or *product space kernel*. The corresponding kernel matrix can be conveniently represented as the unweighted sum of the base kernel matrices:

$$\hat{K}=\sum_{p=1}^{m}K^{p}.$$

Since Kb is a valid kernel matrix itself, we could use Kb as input for Algorithm 3. As a result, the algorithm would learn a kernel from the family

$$\mathcal{K}_{1}=\left\{\left(\sum_{p=1}^{m}K^{p}\right)W\left(\sum_{p=1}^{m}K^{p}\right)\ :\ W\succeq0\right\}$$ $$=\left\{\sum_{p,q=1}^{m}K^{p}WK^{q}\ :\ W\succeq0\right\}.$$
$$(9)$$

## 4.2 Weighted Combination

Note that K1 treats each kernel equally; it is therefore impossible to distinguish *good* features (*i.e.*,
those which can be transformed to best fit C) from bad features, and as a result, the quality of