
Neural Network **C4.5**

Boosting **Boosting**

Data Set Stan Simp Bag Arc Ada **Stan Bag Arc Ada**

breast-cancer-w 3.4 3.5 3.4 3.8 4.0 **5.0 3.7 3.5 3.5**

credit-a 14.8 13.7 13.8 15.8 15.7 **14.9 13.4 14.0 13.7**

credit-g 27.9 24.7 24.2 25.2 25.3 **29.6 25.2 25.9 26.7**

diabetes 23.9 23.0 22.8 24.4 23.3 **27.8 24.4 26.0 25.7**

glass 38.6 35.2 33.1 32.0 31.1 **31.3 25.8 25.5 23.3** heart-cleveland 18.6 17.4 17.0 20.7 21.1 **24.3 19.5 21.5 20.8**

hepatitis 20.1 19.5 17.8 19.0 19.7 **21.2 17.3 16.9 17.2**

house-votes-84 4.9 4.8 4.1 5.1 5.3 **3.6 3.6 5.0 4.8**

hypo 6.4 6.2 6.2 6.2 6.2 **0.5 0.4 0.4 0.4**

ionosphere 9.7 7.5 9.2 7.6 8.3 **8.1 6.4 6.0 6.1** iris 4.3 3.9 4.0 3.7 3.9 **5.2 4.9 5.1 5.6**

kr-vs-kp 2.3 0.8 0.8 0.4 0.3 **0.6 0.6 0.3 0.4**

labor 6.1 3.2 4.2 3.2 3.2 **16.5 13.7 13.0 11.6**

letter 18.0 12.8 10.5 5.7 4.6 **14.0 7.0 4.1 3.9**

promoters-936 5.3 4.8 4.0 4.5 4.6 **12.8 10.6 6.8 6.4**

ribosome-bind 9.3 8.5 8.4 8.1 8.2 **11.2 10.2 9.3 9.6**

satellite 13.0 10.9 10.6 9.9 10.0 **13.8 9.9 8.6 8.4** segmentation 6.6 5.3 5.4 3.5 3.3 **3.7 3.0 1.7 1.5**

sick 5.9 5.7 5.7 4.7 4.5 **1.3 1.2 1.1 1.0**

sonar 16.6 15.9 16.8 12.9 13.0 **29.7 25.3 21.5 21.7**

soybean 9.2 6.7 6.9 6.7 6.3 **8.0 7.9 7.2 6.7**

splice 4.7 4.0 3.9 4.0 4.2 **5.9 5.4 5.1 5.3** vehicle 24.9 21.2 20.7 19.1 19.7 **29.4 27.1 22.5 22.9**

Table 2: Test set error rates for the data sets using (1) a single neural network classifier; (2)
an ensemble where each individual network is trained using the original training set and thus only differs from the other networks in the ensemble by its random initial weights; (3) an ensemble where the networks are trained using randomly resampled training sets (Bagging); an ensemble where the networks are trained using weighted resampled training sets (Boosting) where the resampling is based on the (4) Arcing method and (5) Ada method; (6) a single decision tree classifier;
(7) a Bagging ensemble of decision trees; and (8) Arcing and (9) Ada Boosting ensembles of decision trees.
component classifier at the 95% confidence level; however, none of the ensemble methods are significantly better than any other ensemble approach at **the 95% confidence level.**
To better analyze Table 2's results, Figures 3 and 4 plot the percentage reduction in error for the Ada-Boosting, Arcing, and Bagging method as a function of the original error rate. Examining these figures we note that many of the gains produced by the ensemble methods are much larger than the standard deviation values. **In terms of comparisons**
of different methods, it is apparent from both figures that the Boosting methods (Ada-