ttention-matrix-based loss, neuron selectivity transfer, …</p></li>
<li><p>Freely adding intermediate features matching losses</p></li>
<li><p>Multi-teacher distillation</p></li>
<li><p>…</p></li>
</ul>
<p><strong>TextBrewer</strong> includes:</p>
<ol class="simple">
<li><p><strong>Distillers</strong>: the cores of distillation. Different distillers perform different distillation modes. There are GeneralDistiller, MultiTeacherDistiller, BasicTrainer, etc.</p></li>
<li><p><strong>Configurations and prese