/transformers/mha.html">self-attention layer</a> in <a href="https://nn.labml.ai/transformers/models.html">our transformer implementation</a>. So it&#x27;s just a couple of lines of code, transposing the tensor to apply the MLP across the sequence dimension.</p>
<p>Although the paper applied MLP Mixer on vision tasks, we tried it on a <a href="https://nn.labml.ai/transformers/mlm/index.html">masked language model</a>. <a href="https://nn.labml.ai/transformers/mlp_mixer/experiment.html">Here is the experimen