abs.nlp.annotators.common.IndexedToken">IndexedToken</a>]</span>
      </span>
      
      <p class="shortcomment cmt"></p><div class="fullcomment"><div class="comment cmt"></div><dl class="paramcmts block"><dt class="param">sentence</dt><dd class="cmt"><p>
  input Sentence which can be a full sentence or just a token in type of Sentence</p></dd></dl><dl class="attributes block"> <dt>Definition Classes</dt><dd>BasicTokenizer</dd></dl></div>
    </li><li name="com.johnsnowlabs.nlp.annotators.tokenizer.bpe.B