>This is a <a href="https://pytorch.org">PyTorch</a> implementation of the paper <a href="https://arxiv.org/abs/2109.08668">Primer: Searching for Efficient Transformers for Language Modeling</a>.</p>
<p>The authors do an evolutionary search for transformer architectures. They name the architecture found using the search Primer (PRIMitives searched transformER). <strong>Primer EZ</strong> is the architecture with the two most robust modifications in Primer compared to  the original transformer. Primer EZ tra