Proceedings of the 9th WSEAS International Conference on Artificial Intelligence, Knowledge Engineering and Data Bases (AIKED), University of Cambridge, UK, 2010, edited by L. A. Zadeh et al, pp 191 - 195

$$p(y\mid\mathbf{x},\mathbf{D})\approx\sum_{i=1}^{N}p(y\mid\mathbf{x},\mathbf{\theta}^{i}\mathbf{D})p(\mathbf{\theta}^{i}\mid\mathbf{D})=\frac{1}{N}\sum_{i=1}^{N}p(y\mid\mathbf{x},\mathbf{\theta}^{i},\mathbf{D}).$$

 The above integral can be approximated by using Markov Chain Monte Carlo (MCMC) technique as described in [7], [9]. To perform such an approximation, we need to run a Markov Chain until it has converged to a stationary distribution. Then we can collect N random samples from the posterior p(T | D) to calculate the desired predictive posterior density.

 Using DTs for the classification, we need to find the probability Mtj with which an input x is assigned by terminal node t = 1, …, k to the jth class, where k is the number of terminal nodes in the DT.

 The DT parameters are defined by si pos, si var, si rule, i =
1, …, k - 1, where si pos, si var, and si rule define the position, predictor and rule of each splitting node, respectively. For these parameters the priors can be specified as follows. First, we can define a maximal number of splitting nodes, smax = n - 1. Second we draw any of the m attributes from a uniform discrete distribution U(1, …, m) and assign
{1,..., }
var si  m .

 Finally the candidate value for the splitting variable xj = si var can be drawn from a discrete distribution U(xj(1), …, xj(L)), where L is the number of possible splitting rules for variable xj.

Such priors allow us to explore DTs which split data in as many ways as possible. However the DTs with different numbers of splitting nodes should be explored in the same proportions [7], [9]. To sample DTs of a variable dimensionality, the MCMC technique exploits the Reversible Jump extension. To implement the RJ MCMC technique, in [7], [9] it has been suggested exploring the posterior probability by using the following types of moves:
 Birth. Randomly split the data points falling in one of the terminal nodes by a new splitting node with the variable and rule drawn from the corresponding priors. Death. Randomly pick a splitting node with two terminal nodes and assign it to be one terminal with the united data points. Change-split. Randomly pick a splitting node and assign it a new splitting variable and rule drawn from the corresponding priors.

 Change-rule. Randomly pick a splitting node and assign it a new rule drawn from a given prior. The first two moves, birth and death, are reversible and change the dimensionality of T. The remaining moves provide jumps within the current dimensionality of T. Note that the change-split move is included to make "large" jumps which potentially increase the chance of sampling from a maximal posterior whilst the change-rule move does "local" jumps.

 The RJ MCMC technique starts drawing samples from a DT consisting of one splitting node whose parameters were randomly assigned within the predefined priors. So we need to run the Markov Chain while a DT grows and its likelihood is unstable. This phase is said *burn-in* and it should be preset sufficiently long in order to stabilize the Markov Chain. When the Markov Chain will be enough stable, we can start sampling. This phase is said post *burn-in*.

## 3 The Proposed Method

To test the assumption made in section 2 and refine DT model ensembles obtained with BMA, we propose a new strategy aiming at discarding the DT
models which use weak attributes. According to this strategy, first the BMA technique described in section 2 is used to collect DT models. Then posterior probabilities of using attributes in the ensemble of DT models are estimated. These estimates give us the posterior information on feature importance. Having obtained a range of the posterior probabilities, we then define a threshold value to cut off the attributes with the probabilities below this threshold - we define such attributes as weak. At the next stage we find the DT models which use these weak attributes and finally discard these DT models from the ensemble. Obviously, the larger the threshold value, the greater number of attributes is defined as weak, and therefore the larger portion of DT models is discarded. The efficiency of this discarding technique is evaluated in terms of the accuracy of the refined DT ensemble on the test data. The uncertainty in the ensemble outcomes is evaluated in terms of entropy. Having a set of the threshold probability values obtained in a series of experiments, we can expect that there is an optimal threshold value at which the performance becomes higher. We can also expect to find a threshold value at which the uncertainty becomes lower. In the following section we test the proposed technique on the problem of assessment of newborn brain maturity from sleep EEG.

## 4 Experiments

In our experiments we used EEG data recorded from 686 newborns during sleep hours. The newborns