  Modern machine learning has achieved impressive prediction performance, but
often sacrifices interpretability, a critical consideration in high-stakes
domains such as medicine. In such settings, practitioners often use highly
interpretable decision tree models, but these suffer from inductive bias
against additive structure. To overcome this bias, we propose Fast
Interpretable Greedy-Tree Sums (FIGS), which generalizes the CART algorithm to
simultaneously grow a flexible number of trees in summation. By combining
logical rules with addition, FIGS is able to adapt to additive structure while
remaining highly interpretable. Extensive experiments on real-world datasets
show that FIGS achieves state-of-the-art prediction performance. To demonstrate
the usefulness of FIGS in high-stakes domains, we adapt FIGS to learn clinical
decision instruments (CDIs), which are tools for guiding clinical
decision-making. Specifically, we introduce a variant of FIGS known as G-FIGS
that accounts for the heterogeneity in medical data. G-FIGS derives CDIs that
reflect domain knowledge and enjoy improved specificity (by up to 20% over
CART) without sacrificing sensitivity or interpretability. To provide further
insight into FIGS, we prove that FIGS learns components of additive models, a
property we refer to as disentanglement. Further, we show (under oracle
conditions) that unconstrained tree-sum models leverage disentanglement to
generalize more efficiently than single decision tree models when fitted to
additive regression functions. Finally, to avoid overfitting with an
unconstrained number of splits, we develop Bagging-FIGS, an ensemble version of
FIGS that borrows the variance reduction techniques of random forests.
Bagging-FIGS enjoys competitive performance with random forests and XGBoost on
real-world datasets.
