  In assessing prediction accuracy of multivariable prediction models, optimism
corrections are essential for preventing biased results. However, in most
published papers of clinical prediction models, the point estimates of the
prediction accuracy measures are corrected by adequate bootstrap-based
correction methods, but their confidence intervals are not corrected, e.g., the
DeLong's confidence interval is usually used for assessing the C-statistic.
These naive methods do not adjust for the optimism bias and do not account for
statistical variability in the estimation of parameters in the prediction
models. Therefore, their coverage probabilities of the true value of the
prediction accuracy measure can be seriously below the nominal level (e.g.,
95%). In this article, we provide two generic bootstrap methods, namely (1)
location-shifted bootstrap confidence intervals and (2) two-stage bootstrap
confidence intervals, that can be generally applied to the bootstrap-based
optimism correction methods, i.e., the Harrell's bias correction, 0.632, and
0.632+ methods. In addition, they can be widely applied to various methods for
prediction model development involving modern shrinkage methods such as the
ridge and lasso regressions. Through numerical evaluations by simulations, the
proposed confidence intervals showed favourable coverage performances. Besides,
the current standard practices based on the optimism-uncorrected methods showed
serious undercoverage properties. To avoid erroneous results, the
optimism-uncorrected confidence intervals should not be used in practice, and
the adjusted methods are recommended instead. We also developed the R package
predboot for implementing these methods (https://github.com/nomahi/predboot).
The effectiveness of the proposed methods are illustrated via applications to
the GUSTO-I clinical trial.
