Bayesian MCMC and DT selection techniques within n-fold cross validation in order to estimate the contribution of each feature to the classification. As a result, the contributions of all features can be ranked, and a modeler can find the most critical features in an application.

Related to the STCA problem, the application scenario used in our work is described as follows. The domain experts have defined 12 features listed in Table 1. 

From the collected 2500 data samples, 1250 were used for training and the remaining 1250 for testing. *A priori* information was not available in our case and, therefore, priors were given as "uninformative" except for a Gaussian for the proposal distribution with the variance set to 0.3. The number pmin = 15 was experimentally found to provide the best performance. The numbers of burn-in and post burn-in samples were set equal to 100 k and 10 k, respectively. The proposal probabilities for the death, birth, change-split and change-rules were set to be 0.1, 0.1, 0.2, and 0.6, respectively. Every 7th DT was collected during the post burn-in phase, i.e., the sampling rate was 7. The convergence of the Markov Chain can be visually observed from the top right plot in Fig. 2 - from this plot we can see that after approximately 1/3 of burn-in samples the values of log likelihood do not change significantly. The acceptance level during the post burn-in phase was obtained equal to 0.22 that is close to 0.25 when the diversity of DTs is optimal. The level of confident classification, 0, was predefined to be 0.99. Fig 4 represents the machine diagram of the resultant DT 
selected under the given 0. Finally, the performance of the resultant DT is compared with the performances of the Bayesian DT technique as well as the MAP DT within the 5 fold cross-validation as shown in Fig. 5.

## 6. Conclusion

For estimating uncertainty of decisions in safety-critical engineering applications, we have suggested the Bayesian averaging over decision models using a new strategy of the RJ MCMC sampling for the cases when a *priori* information on the favored structure of models is unavailable. The use of DT models assists experts to interpret causal relations and find factors to account for the uncertainty. However, the Bayesian averaging over DTs allows experts to estimate the uncertainty accurately when a *priori* information on favored structure of DTs is available. 

To interpret an ensemble of diverse DTs sampled by the RJ MCMC technique, experts select the single DT model that has maximum *a posteriori* probability. However in practice this selection technique tends to choose over-fitted DTs which are incapable of providing a high predictive accuracy. 

In this Chapter we have proposed a new procedure of selecting a single DT. This procedure is based on the estimates of uncertainty in the ensemble of the Bayesian DTs. For estimating the uncertainty, the use of an Uncertainty Envelope technique has been advocated. As a result, in our experiments with the STCA data, the suggested technique outperforms the existing Bayesian techniques in terms of predictive accuracy.

Thus, we conclude that the technique proposed for interpreting the ensemble of DTs allows experts to select a single DT providing the most confident estimates of outcomes. These are very desirable properties for classifiers used in safety-critical systems, in which assessment of uncertainty of decisions is of crucial importance. 