stylistic:tone, linguistic:translation, stylistic:style_transfer
Description

for a small enough step size or learning rate η ∈ R + {\displaystyle \eta \in \mathbb {R} _{+}}, then f (a n) ≥ f (a n + 1) {\displaystyle f(\mathbf {a_{n}})\geq f(\mathbf {a_{n+1}})}. In other words, the term η ∇ f (a) {\displaystyle \eta \nabla f(\mathbf {a})} is subtracted from a {\displaystyle \mathbf {a} } because we want to move against the gradient, toward the local minimum. With this observation in mind, one starts with a guess x 0 {\displaystyle \mathbf {x} _{0}} for a local minimum of f {\displaystyle f}, and considers the sequence x 0, x 1, x 2, … {\displaystyle \mathbf {x} _{0},\mathbf {x} _{1},\mathbf {x} _{2},\ldots } such that