number of minimal data instances allowed in DT nodes was 3; the acceptance rate was around 0.25. 

Having obtained the ensemble of DTs, we estimated the importance of all 16 variables for the prediction. The estimates were calculated as the posterior probabilities of variables used in the DTs ensemble as shown in Fig. 1.

| Table 1 Screening tests and attributes of the Trauma data   |                                         |               |
|-------------------------------------------------------------|-----------------------------------------|---------------|
| No                                                          | Screening Tests and Attributes          | Type          |
| 1                                                           | Age                                     | Continuous    |
| 2                                                           | Gender: Male = 1, Female = 0.           | 0,1           |
| 3                                                           | Injury type: Blunt = 1, penetrating = 0 | 0,1           |
| 4                                                           | Head injury, no injury = 0              | 0,1,2,3,4,5,6 |
| 5                                                           | Facial injury                           | 0,1,2,3,4     |
| 6                                                           | Chest injury                            | 0,1,2,3,4,5,6 |
| 7                                                           | Abdominal or pelvic contents injury     | 0,1,2,3,4,5   |
| 8                                                           | Limbs or bony pelvis injury             | 0,1,2,3,4,5   |
| 9                                                           | External injury                         | 0,1,2,3       |
| 10                                                          | Respiration rate                        | Continuous    |
| 11                                                          | Systolic blood pressure                 | Continuous    |
| 12                                                          | Glasgow coma score (GCS) eye response   | 0,1,2,3,4     |
| 13                                                          | GCS motor response                      | 0,1,2,3,4,5,6 |
| 14                                                          | GCS verbal response                     | 0,1,2,3,4,5   |
| 15                                                          | Oximetry                                | Continuous    |
| 16                                                          | Heart rate                              | Continuous    |
| 17                                                          | Died = 1, living = 0.                   | 0,1           |

![0_image_0.png](0_image_0.png)

From Fig. 1 we can observe that the posterior probability of variable 9 is the smallest, around 0.005, while the maximal value is around 0.16 for variable 6. Therefore we can assume that the variable 9 makes negligible contribution to the ensemble's outcome. 

To test our assumptions, we aim to discard this variable from the Trauma data. Table 2 shows the maximal values of loglikelihoods calculated within 5-fold cross-validation for two sets including 16 and 16\9 variables. From this table, we can observe that the loglikelihood value for the 16\9 set becomes greater than that for the set of all 16 variables.

However the performance of the ensemble using the set of 16\9 variables is slightly fewer than that using the set of 16 variables. This can happen because the ensemble using the set of 16\9 variables becomes more overfitted to the training data. Thus, we can conclude that the weakest variable 9 provides better conditions for mitigating the DT ensemble overfitting. 

Table 2 Maximal loglikelihoods, L16 and L16\9, performances and entropies of the ensembles using sets of 16 and 16\9 variables within 5-fold crossvalidation

| Fold           | Loglikelihood (L16)   | Loglikelihood (L16\9)   |
|----------------|-----------------------|-------------------------|
| 1              | -36.14                | -33.4                   |
| 2              | -44.29                | -37.33                  |
| 3              | -37.23                | -33.19                  |
| 4              | -36.24                | -38.37                  |
| 5              | -37.98                | -40.41                  |
| Loglikelihood  | -38.37 ±3.39          | -36.48 ±3.23            |
| Performance, % | 84.66 ±4.70           | 83.85 ±6.95             |
| Entropy        | 29.8 ±2.1             | 30.0 ±4.7               |

## C. Selection Of Dt Ensemble

As shown above, the presence of the weakest variable has the positive effect on mitigating overfitting of the DT ensemble. This means that the DT ensemble should use all 16 input variables during sampling, but then we can exclude those DTs which use the weakest variable 9. After such selection of DTs there is no need to use the variable 9. 

In our experiments this technique was tested within 5fold cross-validation and results shown in Table 3 which compares the performance of the original DT ensemble using all 16 variables with the performance of the selected ensemble. This table also shows the number of DTs omitted after the selection. 

| Original ensemble   | Selected ensemble   |         |                       |                         |       |
|---------------------|---------------------|---------|-----------------------|-------------------------|-------|
| Fold                | Performance,  %     | Entropy | BDTs                  | Performance,  % Entropy |       |
| omitted             |                     |         |                       |                         |       |
| 1                   | 85.93               | 26.47   | 314                   | 85.93                   | 26.46 |
| 2                   | 80.95               | 28.89   | 467                   | 80.95                   | 28.91 |
| 3                   | 84.13               | 31.80   | 217                   | 84.13                   | 31.79 |
| 4                   | 82.54               | 32.05   | 631                   | 82.54                   | 32.04 |
| 5                   | 87.30               | 31.44   | 336                   | 87.30                   | 31.46 |
| 84.17±2.54          | 30.13±2.40          | 393±160 | 84.17±2.54 30.13±2.40 |                         |       |

This table show that the performance of the selected ensemble using 16\9 variables is the same as that of the 3