
| Dataset   | #SE   | Best CV   | Majority   | MLR   |
|-----------|-------|-----------|------------|-------|
| Horse     | 15.0  |           |            |       |
| Splice    | 3.8   |           |            |       |
| Abalone   | 38.3  |           |            |       |
| Led       | 3.3   |           |            |       |
| Credit    | 1 6.1 |           |            |       |
| Nettalk s | 1.5   |           |            |       |
| Coding    | 20.7  |           |            |       |
| Wave form | 16.8  |           |            |       |
| Euthyroid | 1.9   | 1.9       |            |       |
| Vowel     | 2.5   |           |            |       |

## 3.3 How Does Stacked Generalization Compare To Majority Vote?

Let us now compare the error rate of M ′ derived from MLR to that of majority vote a simple decision combination method which requires neither cross validation nor level Table   shows the average error rates of BestCV  majority vote and MLR
learning In order to see whether the relative performances of level   generalizers have any effect on these methods the number of standard errors \#SE between the error rates of the worst performing level   generalizer and BestCV is given  and the datasets are re ordered according to this measure  Since BestCV almost always selects the best performing level generalizer  small values of \#SE indicate that the level   generalizers perform comparably to one another  and vice versa MLR compares favorably to majority vote  with eight wins versus two losses  Out of the eight wins  six have significant differences  the two exceptions are for the Splice and datasets whereas both losses for the Horse and Credit datasets have insignificant Led differences Thus the extra computation for cross validation and level learning seems to have paid off It is interesting to note that the performance of majority vote is related to the size of
\#SE Majority vote compares favorably to BestCV in the first seven datasets where the values of \#SE are small  In the last three where \#SE is large majority vote performs worse This indicates that if the level generalizers perform comparably it is not worth using cross validation to determine the best one  because the result of majority vote    which is far cheaper   is not significantly different  Although small values of \#SE are a necessary condition for majority vote to rival BestCV  they are not a sufficient condition   see Matan for an example  The same applies when majority vote is compared with MLR  MLR
performs significantly better in the five datasets that have large \#SE values  but in only one of the other cases