is natural
for regression. For classification this is not as clear. For
binary classification you use log-loss, or rather you apply
the logistic function to get a binary prediction, for
multi-class you can use 1 vs rest.

So we're sequentially building up a model using what's
called "weak learners", small trees, and create a more
powerful composite model.

---
# Gradient Boosting Algorithm



`$$ f_{1}(x) \approx y  $$`

`$$ f_{2}(x) \approx y - \gamma f_{1}(x) $$`

`$$ f_{3}(x) \approx y - \gamma f_{1}(x) 