
</li>
<li><p><strong>direct utility estimation</strong>: treat states independently</p>
<ul class="simple">
<li><p>run trials to sample utility</p></li>
<li><p>average to get expected total reward for each state = expected total reward from each state</p></li>
</ul>
</li>
<li><p><strong>adaptive dynamic programming</strong> (ADP) - 2 steps</p>
<ul>
<li><p>sample to estimate transition model <span class="math notranslate nohighlight">\(P(s'|s, a)\)</span> and rewards <span class="math notranslate nohighligh