an alphabet containing different types of characters: alphabetic, numeric, punctuation, etc. We use the token's character types to assign it to one or more **syntactic categories:** alphabetic, numeric, etc. These categories form a hierarchy depicted in Fig. 3, where the arrows point from more general to less general categories. A **unique specific token type is** created for every string that appears in at least k **examples, as determined in a preprocessing**
step. The hierarchical representation allows for multi-level generalization. Thus, the token "Boulevard" belongs to the general token types Alphanum **(alphanumeric strings),** Alpha (alphabetic strings), Upper **(capitalized words), as well as to the specific type representing** the string "Boulevard". This representation is flexible and may be expanded to include domain specific information. For example, the numeric type is **divided into categories** that include range information about the number - Large **(larger than 1000),** Medium (medium numbers, between 10 and 1000) and Small **(smaller than 10)— and number of**
digits: 1−, 2−, and 3−digit. Likewise, we may explicitly include knowledge about **the type**
of information being parsed, *e.g.***, some 5-digit numbers could be represented as** zipcode.

![0_image_0.png](0_image_0.png)

We have found that a sequence of specific and general token types is more useful for describing the content of information than the character-level finite state representations used in previous work (Carrasco & Oncina, 1994; Goan et al., 1996). The character-level description is far too fine grained to compactly describe data **and, therefore, leads to poor**
generality. The coarse-grained token-level representation **is more appropriate for most Web** data types. In addition, the data representation schemes used in previous work attempt to describe the entire data field, while we use only the starting and ending sequences, or patterns, of tokens to capture the structure of the data fields. The reason for this is similar to the one above: using the starting and ending patterns allows us to generalize the structural information for many complex fields which have **a lot of variability. Such** fields, *e.g.***, addresses, usually have some regularity in how they start and end that we** can exploit. We call the starting and ending patterns collectively a *data prototype***. As** an example, consider a set of street addresses in Fig. 2. All of the examples start with a