Why are residuals Normally distributed?

When they assume the $$\varepsilon_i$$ have a normal distribution, with mean $$0$$ and variance $$\sigma^2,$$ then the estimated errors $$\hat\varepsilon$$ have a normal distribution with mean $$0$$ and variance $$\operatorname{Var}(\hat\varepsilon).$$

But what is the connection? Why, when errors are normally distributed, the estimated residuals also are normally distributed?

• Because this is a statistics site and we have been around a fairly long time, you can find a great deal written about this with a site search. The key phrase "hat matrix" is especially helpful.
– whuber
Commented Jul 13, 2022 at 13:36

The connection is made through a specific procedure: namely, ordinary least squares. When you suppose a response $$Y$$ is a random variable related to other variables $$X$$ in the form

$$Y = X\beta + \varepsilon$$

where $$\varepsilon$$ is assumed to have a joint Normal distribution (plus some other assumptions that don't matter here), then the least squares solution is

$$\hat\beta = (X^\prime X)^{-}X^\prime Y = (X^\prime X)^{-}X^\prime (X\beta + \epsilon) = \beta + J\varepsilon$$

for a matrix $$J = (X^\prime X)^{-}X^\prime$$ that is computed only from $$X.$$ The residuals, or estimated errors, are the differences

$$\hat\varepsilon = Y - \hat Y = (X\beta + \varepsilon) - X\hat\beta = \varepsilon - XJ\varepsilon = (\mathbb I - H)\varepsilon$$

where $$H = XJ = X(X^\prime X)^{-}X^\prime$$ is the "hat matrix." This exhibits the residuals as linear combinations of $$\varepsilon$$ (with coefficients given by $$\mathbb I - H$$). Linear combinations of jointly Normal variables are Normal, QED.
{"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 0, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0, "katex": 0, "math-container": 17, "wp-katex-eq": 0, "align": 0, "equation": 0, "x-ck12": 0, "texerror": 0}