  Cross-validation is the standard approach for tuning parameter selection in
many non-parametric regression problems. However its use is less common in
change-point regression, perhaps as its prediction error-based criterion may
appear to permit small spurious changes and hence be less well-suited to
estimation of the number and location of change-points. We show that in fact
the problems of cross-validation with squared error loss are more severe and
can lead to systematic under- or over-estimation of the number of
change-points, and highly suboptimal estimation of the mean function in simple
settings where changes are easily detectable. We propose two simple approaches
to remedy these issues, the first involving the use of absolute error rather
than squared error loss, and the second involving modifying the holdout sets
used. For the latter, we provide conditions that permit consistent estimation
of the number of change-points for a general change-point estimation procedure.
We show these conditions are satisfied for optimal partitioning using new
results on its performance when supplied with the incorrect number of
change-points. Numerical experiments show that the absolute error approach in
particular is competitive with common change-point methods using classical
tuning parameter choices when error distributions are well-specified, but can
substantially outperform these in misspecified models. An implementation of our
methodology is available in the R package crossvalidationCP on CRAN.
