8). The network uses the <a href="https://arxiv.org/pdf/1706.03762.pdf">transformer architecture (Vaswani et al., 2017)</a>, explained by <a href="https://jalammar.github.io/illustrated-transformer/">Alammar (2018)</a> and most recently powering the <a href="https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf">GPT-2 (Radford et al., 2019)</a> and <a href="https://arxiv.org/pdf/2005.14165.pdf">GPT-3 (Brown et al., 2020)</a> systems. The transform