linguistic:translation, information:classification, structural:scholar_edition, structural:expansion
Connection to linear regression

Below we make clear the connection between multi-way ANOVA and linear regression.

Linearly re-order the data so that k {\displaystyle k} -th observation is associated with a response y k {\displaystyle y_{k}} and factors Z k, b {\displaystyle Z_{k,b}} where b ∈ { 1, 2, …, B } {\displaystyle b\in \{1,2,\ldots,B\}} denotes the different factors and B {\displaystyle B} is the total number of factors. In one-way ANOVA B = 1 {\displaystyle B=1} and in two-way ANOVA B = 2 {\displaystyle B=2}. Furthermore, we assume the b {\displaystyle b} -th factor has I b {\displaystyle I_{b}} levels, namely { 1, 2, …, I b } {\displaystyle \{1,2,\ldots,I_{b}\}}. Now, we can one-hot encode the factors into the ∑ b = 1 B I b {\textstyle \sum _{b=1}^{B}I_{b}} dimensional vector v k {\displaystyle v_{k}}.

The one-hot encoding function g b: { 1, 2, …, I b } ↦ { 0, 1 } I b {\displaystyle g_{b}:\{1,2,\ldots,I_{b}\}\mapsto \{0,1\}^{I_{b}}} is defined such that the i {\displaystyle i} -th entry of g b (Z k, b) {\displaystyle g_{b}(Z_{k,b})} is g b (Z k, b) i = { 1 if i = Z k, b 0 otherwise {\displaystyle g_{b}(Z_{k,b})_{i}={\begin{cases}1&{\text{if }}i=Z_{k,b}\\0&{\text{otherwise}}\end{cases}}} The vector v k {\displaystyle v_{k}} is the concatenation of all of the above vectors for all b {\displaystyle b}. Thus, v k = {\displaystyle v_{k}=}. In order to obtain a fully general B {\displaystyle B} -way interaction ANOVA we must also concatenate every additional interaction term in the vector v k {\displaystyle v_{k}} and then add an intercept term. Let that vector be X k {\displaystyle X_{k}}.

With this notation in place, we now have the exact connection with linear regression. We simply regress response y k {\displaystyle y_{k}} against the vector X k {\displaystyle X_{k}}. However, there is a concern about identifiability. In order to overcome such issues we assume that the sum of the parameters within each set of interactions is equal to zero. From here, one can use F -statistics or other methods to determine the relevance of the individual factors.