er is to know
        the set of patterns describing the different tokens you want to recognize
        in the input. To make this a bit more formal, here are some definitions:
      </p>
<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
<li class="listitem">
            A token is a sequence of consecutive characters having a collective meaning.
            Tokens may have attributes specific to the token type, carrying additional
            information about the matched 