Hilbert space
Best approximation

This subsection employs the Hilbert projection theorem. If C is a non-empty closed convex subset of a Hilbert space H and x a point in H, there exists a unique point y ∈ C that minimizes the distance between x and points in C, y ∈ C, ‖ x − y ‖ = dist (x, C) = min { ‖ x − z ‖ | z ∈ C }. {\displaystyle y\in C\,,\quad \|x-y\|=\operatorname {dist} (x,C)=\min {\bigl \{}\|x-z\|\mathrel {\big |} z\in C{\bigr \}}\,.}

This is equivalent to saying that there is a point with minimal norm in the translated convex set D = C − x. The proof consists in showing that every minimizing sequence (d) ⊂ D is Cauchy (using the parallelogram identity) hence converges (using completeness) to a point in D that has minimal norm. More generally, this holds in any uniformly convex Banach space.

When this result is applied to a closed subspace F of H, it can be shown that the point y ∈ F closest to x is characterized by y ∈ F, x − y ⊥ F. {\displaystyle y\in F\,,\quad x-y\perp F\,.}

This point y is the orthogonal projection of x onto F, and the mapping P : x → y is linear (see Orthogonal complements and projections). This result is especially significant in applied mathematics, especially numerical analysis, where it forms the basis of least squares methods.

In particular, when F is not equal to H, one can find a nonzero vector v orthogonal to F (select x ∉ F and v = x − y). A very useful criterion is obtained by applying this observation to the closed subspace F generated by a subset S of H.
combination:two_responses