difficult to learn " One potential problem with these methods is that they may generate classifiers that have been fit to systematic noise Moreover this situation may be difficult to detect Typically to validate a particular classifier or method one does a cross validation over a set of labeled data If all of the data was labeled by the same mechanism then the entire dataset contains the same systematic errors Achieving high accuracy on this type of independent" test set  may mean that the method has done an excellent job at fitting to the systematic noise In the past decade the computational machine learning community has investigated variations of PAC learning that model the type of noise that might occur in a real learning environment Angluin Laird Sloan Decat ur More recently  models of non uniform classification noise  Sloan and partial non uniform noise  Decatur have been introduced These models do not assume that each instance has the same misclassification rate and therefore are more realistic models of the types of noise observed in real world applications A recent innovation is to alter the learning procedure for the known noise rates  Decatur However in most real world scenanos one will not have access to the true noise rates of the vanous classes In these cases Decatur s uggests ss one has noise free data with which to evaluate the results of the search

## 3. Filtering Training Data

This section describes a general procedure for identifying mislabeled instances in a training set The first step is to identify candidate instances by using m learning algorithms called filter algorithms to tag instances as correctly or incorrectly labeled To this end a n fold cross validation is performed over the training data For each of the n parts the m agorithms are trained on the other n - parts The m resulting classifiers are then used to tag each instance in the excluded part as either correct or mislabeled An individual classifier tags an instance as mislabeled if it classifies the instance as belonging to a different class than that given by its training label Note that when n is equal to the total number of tion that raining instances this method differs from Weis berg s outlier detection method only in the test used to determine whether a case is an outlier At the end of the n fold cross validation each instance in the training data has been tagged Using this information the second step is to form a classifier using a new version of the training data for which all of the instances identified as mislabeled are removed Filtering can be based on one or more of the m base level classifiers tags The filtered set of training instances is provided as input to the final learning algorithm The resulting classifier is the end product of the approach Figure depicts the general procedure Specific implementations of this general procedure differ in how the filtering is performed  and in the relationship between the filter algorithm s  and the final learning algorithm s

## 3.1 Single Algorithm Filters

One approach is to use the same learning algorithm to construct both the filter and the final classifier This approach is most similar to removing outliers in regression analysis for which the same model is used to test for outliers and for fitting the final model to the data once the outliers have been removed A related method is that proposed by John fo r