 </div>
            <ul><li><code  class="highlight"><span></span><span class="n">d_k</span></code>
 is the number of channels in each head</li></ul>

        </div>
        <div class='code'>
            <div class="highlight"><pre><span class="lineno">70</span>    <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">d_k</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span 