

![0_image_0.png](0_image_0.png)

CPU hours) in 7 cases. These results suggest, that ACE algorithm is not suitable for systems described with very large number of features.

The Boruta algorithm is overall better than ACE. It has found all relevant attributes in 20 cases (23 in most iterations). Its sensitivity was slightly lower than ACE algorithms in few cases, but on the other hand much better selectivity.

However, it achieved higher F-score than ACE algorithm in all cases, see Figure 2.

The highest score (F=0.93) was obtained for the system described with 125 attributes and 2000 objects.

All algorithms take substantial time to complete for large data sets, see Figure 3.

The ACE algorithm is marginally faster than Boruta for low-dimensional sets, but the execution time increases steeply with the number of attributes.

Already for 500 attributes ACE algorithm took roughly 10 times longer to complete, and it has not completed computations within 350 hours for any system described with 2000 attributes, regardless of the number of objects. The time requirements of the ACE algorithm make it impractical for systems described with large number of attributes.

The results obtained in this section suggest that random forest is in most cases a reliable source of feature ranking. Boruta algorithm, has two advantages over the ACE algorithm that make it more suitable for the all-relevant search. It consistently scored highest in the F-score measure, its execution time is reasonable and the sensitivity is only slightly lower in some cases than that of ACE. One should note, that this data set is a very tough test for a random forest importance ranking, due to the way the trees are constructed and how the importance is measured. In the XOR problem the original attribute is informative only in conjunction with other original attribute and so they are rarely included in the trees, and rarely have high impact on the performance of the individual tree. The linear combinations of original attributes are much more