st-directed
input can not be used: for instance because the data are not separated
by the standard characters or you need a finer control over the handling
of the data.


<h2><a name="interface">INTERFACE</a></h2>
<p>
The module contains three routines and it defines a single derived type
and a few convenient parameters.

<p>
The data type is <em>type(tokenizer)</em>, a derived type that holds all
the information needed for parsing the string. It is initialised via the
<em>set_tokenizer</em> s