eters and training data size. We find that BERT was significantly
undertrained, and can match or exceed the performance of every model
published after it. Our best model achieves state-of-the-art results on
GLUE, RACE and SQuAD. These results highlight the importance of previously
overlooked design choices, and raise questions about the source of recently
reported improvements. We release our models and code.</em></p>
<p>Source of the original code: <a class="reference external" href="https://github.com/pyt