
          <div id="comment" class="fullcommenttop"><div class="comment cmt"><p>Tokenizer for Microsoft Phi-4 model (https://huggingface.co/microsoft/phi-4)</p><p>This class implements the byte pair encoding (BPE) tokenizer logic for the phi-4 model.
</p></div><div class="toggleContainer block">
          <span class="toggle">
            Linear Supertypes
          </span>
          <div class="superTypes hiddenContent"><span class="extype" name="com.johnsnowlabs.nlp.annotators.tokenizer.bpe.BpeTokenizer">