al" href="https://github.com/ymcui/Chinese-BERT-wwm"><strong>RoBERTa-wwm-ext</strong></a> released by the Joint Laboratory of HIT and iFLYTEK Research.</p></li>
</ul>
<p>We have tested different student models. To compare with public results, the student models are built with standard transformer blocks except BiGRU which is a single-layer bidirectional GRU. The architectures are listed below. Note that the number of parameters includes the embedding layer but does not include the output layer of the each s