�� | ————- | ————— |
| DistilBERT (T6)    | 81.6 / 81.1 | 78.1 / 86.2   | -               |
| BERT<sub>6</sub>-PKD (T6)     | 81.5 / 81.0     | 77.1 / 85.3   | -|
| BERT-of-Theseus (T6) | 82.4/  82.1   | -        | -                |
| BERT<sub>3</sub>-PKD (T3)     | 76.7 / 76.3     | -             | -|
| TinyBERT (T4-tiny) | 82.8 / 82.9                | 72.7 / 82.1   | -|</p>
<p>Our results:</p>
<p>| Model (ours) | MNLI  | SQuAD  | CoNLL-2003 |
| :————-  | ————— | ——