ies to find optimal policy</p>
<ul>
<li><p><em>bellman eqn</em>: <span class="math notranslate nohighlight">\(U(s) = R(s) + \gamma \: \underset{a}{\max} \sum_{s'} P(s' \vert s, a) U(s')\)</span></p></li>
<li><p>start with arbitrary utilities</p></li>
<li><p>recalculate several times with <em>Bellman update</em> to approximate solns to bellman eqn</p></li>
</ul>
</li>
<li><p>value iteration eventually converges</p>
<ul>
<li><p><em>contraction</em> - function that brings variables together</p>
<ul>
<li><p>con