ext token probabilities</span>
<span class="sd">    topK</span>
<span class="sd">        The number of highest probability vocabulary tokens to keep for</span>
<span class="sd">        top-k-filtering</span>
<span class="sd">    topP</span>
<span class="sd">        Top cumulative probability for vocabulary tokens</span>

<span class="sd">        If set to float &lt; 1, only the most probable tokens with probabilities</span>
<span class="sd">        that add up to ``topP`` or higher are kept for generation.<