  Beta coefficients for linear regression models represent the ideal form of an
interpretable feature effect. However, for non-linear models and especially
generalized linear models, the estimated coefficients cannot be interpreted as
a direct feature effect on the predicted outcome. Hence, marginal effects are
typically used as approximations for feature effects, either in the shape of
derivatives of the prediction function or forward differences in prediction due
to a change in a feature value. While marginal effects are commonly used in
many scientific fields, they have not yet been adopted as a model-agnostic
interpretation method for machine learning models. This may stem from their
inflexibility as a univariate feature effect and their inability to deal with
the non-linearities found in black box models. We introduce a new class of
marginal effects termed forward marginal effects. We argue to abandon
derivatives in favor of better-interpretable forward differences. Furthermore,
we generalize marginal effects based on forward differences to multivariate
changes in feature values. To account for the non-linearity of prediction
functions, we introduce a non-linearity measure for marginal effects. We argue
against summarizing feature effects of a non-linear prediction function in a
single metric such as the average marginal effect. Instead, we propose to
partition the feature space to compute conditional average marginal effects on
feature subspaces, which serve as conditional feature effect estimates.
