;&quot;&quot;Think about if this would need to work differently in training vs.</span>
<span class="sd">    eval mode, like multiplicative dropout.</span>
<span class="sd">    </span>
<span class="sd">    Work in progress, not sure if xavier normal is a good choice - just an </span>
<span class="sd">    example.</span>
<span class="sd">    </span>
<span class="sd">    Also look into if floating point addition might be faster/slower on gpu</span>
<span class="sd">    than multiplication.</span>
<span class="