ate new samples each time the policy is changed</p></li>
<li><p>off policy - can improve policy without generating new samples from it (so look at a bunch of samples then update)</p></li>
<li><p>offline rl - data collected only once with any policy, then want to learn good policy from that</p></li>
</ul>
</li>
</ul>
<p><img alt="rl_sample_efficiency" src="../../_images/rl_sample_efficiency.png" /></p>
</section>
<section id="other-problems">
<h3><span class="section-number">1.10.6.3. </span>other problems<a