

![0_image_0.png](0_image_0.png)

example, the 15% noise measured at segment level reduces to 5.6% at position level. Since Node decomposes the clique over positions, it can ignore wrong positions during optimization and be more robust against noisy cliques. We corroborate this in Figure 4(a), where for each of the 58 datasets, we plot the difference in F1 of Clique and Node versus clique noise in the dataset's agreement set. We observe that whenever Clique performs sufficiently worse than Node there is high noise in the cliques. In low noise settings, Clique is often much better than Node.

## 5.3 Noise In The Agreement Set

As discussed in Section 3, it is important to choose high quality agreement sets. Figure 4(b) shows the F1 scores of Node under three clique generation schemes of varying noise. The rightmost points are for the conventional practice of choosing arbitrary unigram repetitions as cliques and has a noise of 17.3% at position level. The middle point is our method of clique generation where we reduce the noise to 5.6% and the leftmost are ideal cliques with zero noise obtained by using the ground truth to remove all noisy unigrams. We find that our clique selection method enjoys accuracy very close to that with noise-free cliques and the accuracy with carelessly chosen cliques is much lower.

## 5.4 Comparison With Em-Based Approaches

In Section 4.1 we described how the PR framework [11] is applicable to our problem. We show its results in the last column of Table 2. The accuracy of PR is comparable to the Clique method showing that distance-based and likelihood terms serve similar goals in our setting. However, the PR approach is more than four times slower than our likelihood objective maximized using gradient ascent. The PR objective requires the EM algorithm for training. In typical feature-based structured models, the M-step tends to be expensive and it is best not wasted on working with fixed E-values. To evaluate the tradeoffs between EM and gradient-based training we also ran the EM algorithm of [17] whose gradient-based version we call TR1 in Table 2. We found the EM trainer (not shown) to have an F1 0.4% less than TR1 and also a factor of two slower.

## 6 Conclusion

We presented a framework for jointly training multiple extraction models exploiting partial content overlap across sources. Partial overlap opens up a slew of problems - choosing a noise-free agreement set, a training objective or its approximation, and an optimization algorithm. We showed that while decomposing the agreement term over cliques provides a tractable yet accurate method of agreement, it also turns out to be more robust against clique noise than methods that approximate the fused graph. We also presented a strategy for computing clean agreement sets that is far superior to the na¨ıve alternative. Through extensive experiments on various real datasets we showed that our agreement-term decompositions on cliques and positions are more robust, accurate, and faster than alternatives like multi-view learning.