

![0_image_0.png](0_image_0.png)

difference between the two terms. But a difference arises in the asymmetric case when one model is strong and peaked and the other is weak and flat. One possible maxima for the two-view term would be the strong model flattening out and becoming identical to the weaker one. As our agreement term is averse to flat marginals, it will avoid this maxima. Figure 4.1 illustrates the difference between the two terms for two binomial distributions.

Given such a relationship between the two terms, we compare the performance of our algorithms with the multi-view algorithm in Section 5. The multi-view objective will be optimized via EM because the gradient cannot be computed tractably for structured models.

## 4.2 Label Transfer

Another set of approaches deal with transferring labeled data from one source to the other. One such inexpensive approach is an asymmetric staged strategy of training the model for a more confident source first, transferring its certain labels to the next source and so on. This requires a good ordering of sources to control error-cascades. In Section 5, we show that even with suitable heuristics, this scheme suffers from huge performance deviations.

Similar label transfer ideas have been employed in training rule-based extraction wrappers [5].

More sophisticated methods in this class include CoBoosting [7], Co-Training [1], and the two-view Perceptron [2] that train two models in tandem by each model providing labeled data for the other. A detailed comparison of these models in [10] show that these methods are less robust than methods that jointly train all models.

## 4.3 Inference-Only Approaches

Another option is to only train the base models, and perform any corrections at runtime through collective inference. Such strategies have been used on a variety of NLP tasks [21, 8, 15]. These methods usually end up using cliques only over unigrams, with little focus on controlling their noise. The most common practice is marking arbitrary repetitions of a token as a clique. As we show in Section 5, our collective training algorithms are significantly better than collective inference, even with identical agreement sets. A prime limitation of inference-only approaches is that they cannot transfer the benefits of overlap to other instances which do not overlap.

## 5 Experimental Evaluation

We present extensive experiments over several real datasets covering a rich diversity of data characteristics. Our first set of experiments seek to justify collective training by showing substantial benefits over base models, and alternatives like staged training and collective inference discussed in Sections 4.2 and 4.3 respectively. Second, we study our collective training approach in detail by comparing the accuracies of the various approximations made in Section 2.3. Third, we demonstrate the importance of choosing high quality agreement sets by comparing various set-generation schemes. Finally, we make a case that our simple gradient ascent algorithm is as accurate as existing traditional EM-based approaches [17, 11] while being considerably faster.

Datasets: We use a corpus of 58 real datasets, each comprising multiple HTML lists. All lists in a dataset contain semi-structured instances relevant to a dataset-specific relation e.g. University mottos, Caldecott medal winners, movies by James Cagney, Supreme court cases etc. The 58 datasets exhibit a wide spectrum of behavior in terms of their base accuracy, number of sources, number of cliques per instance, their noise levels,