 class='code'>
            <div class="highlight"><pre><span class="lineno">105</span>        <span class="k">return</span> <span class="n">x</span></pre></div>
        </div>
    </div>
    <div class='section' id='section-19'>
        <div class='docs doc-strings'>
            <div class='section-link'>
                <a href='#section-19'>#</a>
            </div>
            <h2>Multi-DConv-Head Attention (MDHA)</h2>
<p>We extend our original implementation of <a href="../mha.html#MHA">Multi-Head Attent