{
             print STDERR
               &quot;$input_line_number: $token\n&quot;;
         }
     }
 }</code></pre>

<p>This example pulls out these tokenization variables from the $line_of_tokens hash reference:</p>

<pre><code>     $rtoken_type = $line_of_tokens-&gt;{_rtoken_type};
     $rtokens     = $line_of_tokens-&gt;{_rtokens};</code></pre>

<p>The variable <code>$rtoken_type</code> is a reference to an array of token type codes, and <code>$rtokens</code> is a reference to a corresponding array of