# Committee-Based Sample Selection For Probabilistic Classifiers

## Shlomo  Argamon- Engelson

ARG AMON @M AIL  JCT  AC  IL
Department of Computer Science Jerusalem College of Technology, Machon Lev P.O.B. 16031 Jerusalem 91160, Israel

## Ido Dagan

DAGAN © CS  BIU  AC  IL
Department of Mathematics and Computer Science Bar Ilan University 52900 Ramat Gan, Israel

## Abstract

In many real-world learning tasks it is expensive to acquire a sufficient number of labeled examples for training. This paper investigates methods for reducing annotation cost by sample selection . In this approach, during training the learning program examines many ulabeled examples and selects for labeling only those that are most informative at each stage. This avoids redundantly labeling examples that contribute little new information.

Our work follows on previous research on Query By Committee, and extends the committee-based paradigm to the context of probabilistic classification. We describe a family of empirical methods for committee-based sample selection in probabilistic classification models, which evaluate the informativeness of an example by measuring the degree of disagreement between several model variants. These variants (the committee) are drawn randomly from a probability distribution conditioned by the training set labeled so far.

The method was applied to the real-world natural language processing task of stochastic part-of-speech tagging. We find that all variants of the method achieve a significant reduction in annotation cost, although their computational efficiency differs. In particular, the simplest variant, a two member committee with no parameters to tune, gives excellent results. We also show that sample selection yields a significant reduction in the size of the model used by the tagger.

## 1. Introduction

active learning