matic</span>
<span class="sd">    study compares pre-training objectives, architectures, unlabeled data sets,</span>
<span class="sd">    transfer approaches, and other factors on dozens of language understanding</span>
<span class="sd">    tasks. By combining the insights from our exploration with scale and our new</span>
<span class="sd">    Colossal Clean Crawled Corpus, we achieve state-of-the-art results on many</span>
<span class="sd">    benchmarks covering summarization, question answering, text cla