information:highlighting, structural:expansion, information:classification
Boosting as gradient descent

In the gradient descent analogy, the output of the classifier for each training point is considered a point (F t (x 1), …, F t (x n)) {\displaystyle \left(F_{t}(x_{1}),\dots,F_{t}(x_{n})\right)} in n-dimensional space, where each axis corresponds to a training sample, each weak learner h (x) {\displaystyle h(x)} corresponds to a vector of fixed orientation and length, and the goal is to reach the target point (y 1, …, y n) {\displaystyle (y_{1},\dots,y_{n})} (or any region where the value of loss function E T (x 1, …, x n) {\displaystyle E_{T}(x_{1},\dots,x_{n})} is less than the value at that point), in the fewest steps. Thus AdaBoost algorithms perform either Cauchy (find h (x) {\displaystyle h(x)} with the steepest gradient, choose α {\displaystyle \alpha } to minimize test error) or Newton (choose some target point, find α h (x) {\displaystyle \alpha h(x)} that brings F t {\displaystyle F_{t}} closest to that point) optimization of training error.