# Wrapper Maintenance: A Machine Learning Approach

Kristina Lerman lerman@isi.edu USC Information Sciences Institute 4676 Admiralty Way Marina del Rey, CA 90292 USA
Steven N. Minton minton@fetch.com Fetch Technologies 4676 Admiralty Way Marina del Rey, CA 90292 USA
Craig A. Knoblock knoblock@isi.edu USC Information Sciences Institute and Fetch Technologies 4676 Admiralty Way Marina del Rey, CA 90292 USA

## Abstract

The proliferation of online information sources has led to an increased use of wrappers for extracting data from Web sources. While most of the previous research has focused on quick and efficient generation of wrappers, the development of tools for wrapper maintenance has received less attention. This is an important research problem because Web sources often change in ways that prevent the wrappers from extracting data correctly. We present an efficient algorithm that learns structural information about data from positive examples alone. We describe how this information can be used for two wrapper maintenance applications: wrapper verification and reinduction. The wrapper verification system detects when a wrapper is not extracting correct data, usually because the Web source has changed its format. The reinduction algorithm automatically recovers from changes in the Web source by identifying data on Web pages so that a new wrapper may be generated for this source. To validate our approach, we monitored 27 wrappers over a period of a year.

The verification algorithm correctly discovered 35 of the 37 wrapper changes, and made 16 mistakes, resulting in precision of 0.73 and recall of 0.**95. We validated the reinduction algorithm on ten Web sources. We were able to successfully reinduce the wrappers,**
obtaining precision and recall values of 0.90 and 0.**80 on the data extraction task.**

## 1. Introduction

There is a tremendous amount of information available online, but much of this information is formatted to be easily read by human users, not computer applications. Extracting information from semi-structured Web pages is an increasingly important capability for Web-based software applications that perform information management functions, such as shopping agents (Doorenbos, Etzioni, & Weld, 1997) and virtual travel assistants (Knoblock, Minton, Ambite, Muslea, Oh, & Frank, 2001b; Ambite, Barish, **Knoblock, Muslea, Oh, &** Minton, 2002), among others. These applications, often referred to as agents, rely on Web wrappers that extract information from semi-structured **sources and convert it to** a structured format. Semi-structured sources are those that **have no explicitly specified** grammar or schema, but have an implicit grammar that can be used to identify relevant
°c 2003 AI Access Foundation and Morgan Kaufmann Publishers. All rights reserved.