Analysis of our results suggests that the performance of both Boosting methods (AdaBoosting and Arcing) is at least partly dependent on the data **set being examined, where**
Bagging shows much less correlation. The strong correlations for Boosting may be partially explained by its sensitivity to noise, a claim supported by additional tests. Finally, we show that much of the performance enhancement for an ensemble comes with the first few classifiers combined, but that Boosting decision trees may continue to further improve with larger ensemble sizes.

In conclusion, as a general technique for decision trees and **neural networks, Bagging is**
probably appropriate for most problems, but when appropriate, Boosting (either Arcing or Ada) may produce larger gains in accuracy.

## Acknowledgments

This research was partially supported by University of Minnesota Grants-in-Aid to both authors. Dave Opitz was also supported by National Science Foundation grant IRI-9734419, the Montana DOE/EPSCoR Petroleum Reservoir Characterization Project, a MONTS
grant supported by the University of Montana, and a Montana Science Technology Alliance grant. This is an extended version of a paper published in the *Fourteenth National* Conference on Artificial Intelligence.

## References

Ali, K., & Pazzani, M. (1996). Error reduction through learning multiple descriptions.

Machine Learning, 24**, 173–202.**
Alpaydin, E. (1993). Multiple networks for function learning. In Proceedings of the 1993 IEEE International Conference on Neural Networks**, Vol. I, pp. 27–32 San Francisco.**
Arbib, M. (Ed.). (1995). *The Handbook of Brain Theory and Neural Networks***. MIT Press.**
Asker, L., & Maclin, R. (1997a). Ensembles as a sequence of classifiers. In Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence**, pp. 860–865**
Nagoya, Japan.

Asker, L., & Maclin, R. (1997b). Feature engineering and classifier selection: A case study in Venusian volcano detection. In Proceedings of the Fourteenth International Conference on Machine Learning**, pp. 3–11 Nashville, TN.**
Bauer, E., & Kohavi, R. (1999). An empirical comparison of voting classification algorithms:
Bagging, boosting, and variants. *Machine Learning*, 36**, 105-139.**
Baxt, W. (1992). Improving the accuracy of an artificial neural network using multiple differently trained networks. *Neural Computation*, 4**, 772–780.**
Breiman, L. (1996a). Bagging predictors. *Machine Learning*, 24**(2), 123–140.**
Breiman, L. (1996b). Bias, variance, and arcing classifiers. Tech. rep. 460, UC-Berkeley, Berkeley, CA.