 digit */
    ctrlChars = CHR(1) .. CHR(31) . /* ascii control characters */
</PRE>

<P><B>Tokens</B>. A <i>token</i> is a terminal symbol for the parser but a
syntactically structured symbol for the scanner. The token structure has to be
described by a regular expression in EBNF:</P>

<PRE>   Tokens      = &quot;TOKENS&quot; { Token } .
   Token       = TokenSymbol [ &quot;=&quot; TokenExpr &quot;.&quot; ] .
   TokenExpr   = TokenTerm { &quot;|&quot; TokenTerm } .
   TokenTerm   = TokenFactor {TokenFactor}