
## 5.2 Discussion

Why do the minority-class predictions have a higher error rate ( PPV ) than the majority-class predictions ( NPV )? There are at least two reasons. First, consider a classifier trandom where the partitions L are chosen randomly and the assignment of each L ∈ L to LP and LN is also made randomly (recall that LP and LN represent the regions labeled with the positive and negative classes). For a two-class learning problem the expected overall accuracy, αt, of this randomly generated and labeled classifier must be 0.5. However, the expected accuracy of the regions in the positive partition, αLP
, will be ρ while the expected accuracy of the regions in the negative partition, αLN
, will be 1 - ρ. For a highly unbalanced class distribution where ρ=.01, αLP
= .01 and αLN = .99. Thus, in such a scenario the negative/majority predictions will be much more 
"accurate." While this "test distribution effect" will be small for a well-learned concept with a low Bayes error rate (and non-existent for a perfectly learned concept with a Bayes error rate of 0), many learning problems are quite hard and have high Bayes error rates.4 The results in Table 4 suggest a second explanation for why the minority-class predictions are so error prone. According to the coverage results, minority-labeled predictions tend to be formed from fewer training examples than majority-labeled predictions. *Small disjuncts*, which are the components of disjunctive concepts (i.e., classification rules, decision-tree leaves, etc.) that cover few training examples, have been shown to have a much higher error rate than large disjuncts 
(Holte, et al., 1989; Weiss & Hirsh, 2000). Consequently, the rules/leaves labeled with the minority class have a higher error rate partly because they suffer more from this "problem of small disjuncts."
Next, why are minority-class examples classified incorrectly much more often than majorityclass examples (FN > FP)—a phenomenon that has also been observed by others (Japkowicz & 
Stephen, 2002)? Consider the estimated accuracy, at
, of a classifier t, where the test set is drawn from the true, underlying distribution D: 

## At = Tp - Rtest + Tn - (1 - Rtest) [2]

Since the positive class corresponds to the minority class, rtest < .5, and for highly unbalanced data sets rtest << .5. Therefore, false-positive errors are more damaging to classification accuracy than false negative errors are. A classifier that is induced using an induction algorithm geared toward maximizing accuracy therefore should "prefer" false-negative errors over false-positive errors. This will cause negative/majority examples to be predicted more often and hence will lead to a higher error rate for minority-class examples. One straightforward example of how learning algorithms exhibit this behavior is provided by the common-sense rule: if there is no evidence favoring one classification over another, then predict the majority class. More generally, induction algorithms that maximize accuracy should be biased to perform better at classifying majority-class examples than minority-class examples, since the former component is weighted more heavily when calculating accuracy. This also explains why, when learning from data sets with a high degree of class imbalance, classifiers rarely predict the minority class. 

A second reason why minority-class examples are misclassified more often than majorityclass examples is that fewer minority-class examples are likely to be sampled from the distribu-