10.6.3.4. </span>offline rl<a class="headerlink" href="#offline-rl" title="Link to this heading">#</a></h4>
<ul class="simple">
<li><p>core issue with offline RL: want policy that improves over the training policy, but can‚Äôt deviate from the training policy due to distr. shift</p></li>
<li><p>one idea: instead of collecting new samples with policy, reweight samples using importance sampling based on policy</p></li>
<li><p><a class="reference external" href="https://arxiv.org/abs/2110.06169">IQL: Impli