

![0_image_0.png](0_image_0.png)

We assume that input web documents are already downloaded, cleaned, encoded into the UTF-8 character set, and formatted in XML (Nazarenko et al. , 2006). Documents are first tokenized to define offsets to ensure the homogeneity of the various annotations. Then, documents are processed through several modules: named entity recognition, word and sentence segmentation, lemmatization, part-of-speech tagging, term tagging, parsing, semantic tagging and anaphora resolution.

Although this architecture is quite traditional, few points should be highlighted:
Tokenization computes a first basic non-linguistic segmentation of the document, which is used for further reference.  The tokens are the basic textual units in the text processing line.  Tokenization serves no other purpose but to provide a starting point for segmentation. This level of annotation follows the recommendations of the TC37SC4/TEI workgroup, even if we refer to the character offset rather than pointer mark-up (TEI element ptr) in the textual signal to mark the token boundaries.  To simplify further processing, we distinguish different types of tokens: alphabetical tokens, numerical tokens, separating tokens and symbolic tokens.