sion
$$ L(\mathbf{x}_i, y_i, \mathbf{w}, b) = \sum_i(y_i - \hat{y}_i)^2 $$
$$ = \sum_i(y_i - w^T \mathbf{x}_i - b)^2 $$

optimize:

`$$\min_{w \in \mathbb{R}^p, b\in\mathbb{R}} \sum_{i=1}^n (y_i - w^T\mathbf{x}_i - b)^2$$`

gradient descent:
$$w_{j+1} = w_j - \gamma \frac{\partial L(\mathbf{x}_i, y_i, \mathbf{w}, b)}{\partial\mathbf{w}}$$
]
--
.right-column[
## Gradient Boosting
$$ L(y_i, \hat{y}_i) = \sum_i(y_i - \hat{y}_i)^2 $$

optimize:

`$$\min_{\hat{y}\in \mathbb{R}^n} \sum_{i=1}^n (y_i - \hat{y_i})^2