classifier to compensate for changes made to the class distribution of the training set is described and this adjustment is shown to substantially improve classifier accuracy (see Appendix A). We consider this to be particularly significant because previous research on the effect of class distribution on learning has not employed this, or any other, adjustment (Catlett, 1991; Chan & Stolfo, 1998; Japkowicz & Stephen, 2002). 

Practitioners often make changes to the class distribution of training data, especially when the classes are highly unbalanced. These changes are seldom done in a principled manner and the reasons for changing the distribution—and the consequences—are often not fully understood. 

We hope this article helps researchers and practitioners better understand the relationship between class distribution and classifier performance and permits them to learn more effectively when there is a need to limit the amount of training data. 

## Acknowledgments

We would like to thank Haym Hirsh for the comments and feedback he provided throughout this research. We would also like to thank the anonymous reviewers for helpful comments, and IBM 
for a Faculty Partnership Award. 

## Appendix A: Impact Of Class Distribution Correction On Classifier Performance

Table A1 compares the performance of the decision trees labeled using the uncorrected frequencybased estimate (FB) with those labeled using the corrected frequency-based estimate (CT-FB). 

| % Rel.                                                      | % Labels           |       |         |         |      |       |
|-------------------------------------------------------------|--------------------|-------|---------|---------|------|-------|
| Dataset                                                     | FB                 | CT-FB | Improv. | Changed | FB   | CT-FB |
| letter-a                                                    | 9.79               | 5.38  | 45.0    | 39.0    | 2.7  | 7.2   |
| pendigits                                                   | 4.09               | 4.02  | 1.7     | 3.2     | 5.6  | 7.8   |
| abalone                                                     | 30.45              | 22.97 | 24.6    | 5.6     | 8.5  | 19.1  |
| sick-euthyroid                                              | 9.82               | 6.85  | 30.2    | 6.7     | 8.8  | 14.6  |
| connect-4                                                   | 30.21              | 27.57 | 8.7     | 14.7    | 8.5  | 10.4  |
| optdigits                                                   | 6.17               | 3.41  | 44.7    | 42.5    | 6.0  | 21.2  |
| covertype                                                   | 6.62               | 6.46  | 2.4     | 2.4     | 7.0  | 8.5   |
| solar-flare                                                 | 36.20              | 29.12 | 19.6    | 20.4    | 19.3 | 30.7  |
| phone                                                       | 17.85              | 14.81 | 17.0    | 3.2     | 25.2 | 44.4  |
| letter-vowel                                                | 18.89              | 14.16 | 25.0    | 44.1    | 15.9 | 30.2  |
| contraceptive                                               | 40.77              | 39.65 | 2.7     | 11.1    | 20.6 | 27.6  |
| adult                                                       | 22.69              | 20.05 | 11.6    | 30.7    | 19.6 | 36.8  |
| splice-junction                                             | 9.02               | 8.74  | 3.1     | 14.1    | 20.1 | 28.4  |
| network2                                                    | 30.80              | 29.96 | 2.7     | 1.2     | 32.9 | 40.1  |
| yeast                                                       | 34.01              | 28.80 | 15.3    | 4.6     | 29.4 | 47.0  |
| network1                                                    | 31.99              | 30.99 | 3.1     | 1.3     | 32.9 | 38.2  |
| car                                                         | 8.26               | 7.92  | 4.1     | 5.3     | 25.9 | 33.8  |
| german                                                      | 38.37              | 37.09 | 3.3     | 16.1    | 30.8 | 35.8  |
| breast-wisc                                                 | 6.76               | 6.74  | 0.3     | 0.4     | 38.5 | 38.7  |
| blackjack                                                   | 33.02              | 28.71 | 13.1    | 17.1    | 42.9 | 76.2  |
| weather                                                     | 34.62              | 34.61 | 0.0     | 0.0     | 40.5 | 40.5  |
| bands                                                       | 32.68              | 32.68 | 0.0     | 0.6     | 90.2 | 90.2  |
| market1                                                     | 25.77              | 25.77 | 0.0     | 23.9    | 46.0 | 48.6  |
| crx                                                         | 20.84              | 21.48 | -3.1    | 17.2    | 46.2 | 51.4  |
| kr-vs-kp                                                    | 1.22               | 1.22  | 0.0     | 0.2     | 58.5 | 58.5  |
| move                                                        | 28.24              | 28.24 | 0.0     | 20.8    | 52.6 | 60.7  |
| Average                                                     | 21.89              | 19.90 | 10.6    | 13.3    | 28.3 | 36.4  |
| Error Rate                                                  | % Errors from Min. |       |         |         |      |       |
| Table A1: Impact of the Probability Estimates on Error Rate |                    |       |         |         |      |       |
