u could use any model as a
weak learner, but trees just work really well.

---

# Gradient Boosting Algorithm



`$$ f_{1}(x) \approx y  $$`

`$$ f_{2}(x) \approx y - f_{1}(x) $$`

`$$ f_{3}(x) \approx y - f_{1}(x) - f_{2}(x)$$`

--

$y \approx$ ![:scale 22.5%](images/grad_boost_term_1.png) + ![:scale 22.5%](images/grad_boost_term_2.png) + ![:scale 20%](images/grad_boost_term_3.png) + ...
???
Let's look at the regression case first. We start by
building a single tree f1 to try to predict the output y.
But w