><span class="o">.</span><span class="n">key</span></code>
, and <code  class="highlight"><span></span><span class="bp">self</span><span class="o">.</span><span class="n">value</span></code>
.</p>
<p>üìù <em>We feel this cleaner implementation is easier to understand since it clearly shows the difference between this and vanilla transformer multi-head attention</em>. </p>

        </div>
        <div class='code'>
            <div class="highlight"><pre><span class="lineno">127</span>        <span cla