nts？</p>
<p><strong>A</strong>: Knowledge distillation usually requires more training epochs and larger learning rate than training on labeled dataset. For example, training SQuAD on BERT-base usually takes 3 epochs with lr=3e-5; however, distillation takes 30~50 epochs with lr=1e-4. <strong>The conclusions are based on our experiments, and you are advised to try on your own data</strong>.</p>
</div>
<div class="section" id="known-issues">
<h2>Known Issues<a class="headerlink" href="#known-issues" title="