pan></span></span></span></span></p>
<h3>Multi-DConv-Head Attention (MDHA)</h3>
<p>The next effective modification is a depth-wise <span ><span class="katex"><span aria-hidden="true" class="katex-html"><span class="base"><span class="strut" style="height:0.72777em;vertical-align:-0.08333em;"></span><span class="mord">3</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">√ó</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span