le="Link to this heading">#</a></h4>
<ul class="simple">
<li><p><span class="math notranslate nohighlight">\(U(s) = \underset{a}{\max} \: Q(s,a)\)</span></p>
<ul>
<li><p>ADP version: <span class="math notranslate nohighlight">\(Q(s, a) = R(s) + \gamma \sum_{s'} P(s'|s, a) \underset{a'}{\max} Q(s', a')\)</span></p></li>
<li><p>TD version: <span class="math notranslate nohighlight">\(Q(s,a) = Q(s,a) + \alpha [R(s) - Q(s,a) + \gamma \: \underset{a'}{\max} Q(s', a')]\)</span> - <strong>this is what is usually r