information:extraction
Totally corrective algorithms

For steepest descent versions of AdaBoost, where α t {\displaystyle \alpha _{t}} is chosen at each layer t to minimize test error, the next layer added is said to be maximally independent of layer t: it is unlikely to choose a weak learner t+1 that is similar to learner t. However, there remains the possibility that t+1 produces similar information to some other earlier layer. Totally corrective algorithms, such as LPBoost, optimize the value of every coefficient after each step, such that new layers added are always maximally independent of every previous layer. This can be accomplished by backfitting, linear programming or some other method.