Principal component analysis
Derivation using the covariance method

Let X be a d -dimensional random vector expressed as column vector. Without loss of generality, assume X has zero mean.

We want to find (∗) {\displaystyle (\ast)} a d × d orthonormal transformation matrix P so that PX has a diagonal covariance matrix (that is, PX is a random vector with all its distinct components pairwise uncorrelated).

A quick computation assuming P {\displaystyle P} were unitary yields:

Hence (∗) {\displaystyle (\ast)} holds if and only if cov (X) {\displaystyle \operatorname {cov} (X)} were diagonalisable by P {\displaystyle P}.

This is very constructive, as cov(X) is guaranteed to be a non-negative definite matrix and thus is guaranteed to be diagonalisable by some unitary matrix.
keywords:letter_frequency