blic results from <a class="reference external" href="https://arxiv.org/abs/1910.01108">DistilBERT</a>, <a class="reference external" href="https://arxiv.org/abs/1908.09355">BERT-PKD</a>, <a class="reference external" href="https://arxiv.org/abs/2002.02925">BERT-of-Theseus</a>, <a class="reference external" href="https://arxiv.org/abs/1909.10351">TinyBERT</a> and our results below for comparison.</p>
<p>Public results:</p>
<p>| Model (public) | MNLI  | SQuAD  | CoNLL-2003 |
| :————-  | ————�