however that lexers with complex rules like these are more prone to lose
    track of their state, especially if they span multiple lines.</p>

    <p><a id="lexer.Summary"></a></p>

    <h4>Summary</h4>

    <p>Lexers primarily consist of tokens and grammar rules. At your disposal are a
    number of convenience patterns and functions for rapidly creating a lexer. If
    you choose to use predefined token names for your tokens, you do not have to
    define how the lexer highlights them. The tokens will in