Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Model Evaluation, Model Selection, and Algorithm Selection in Machine
  Learning   The correct use of model evaluation, model selection, and algorithm selection
techniques is vital in academic machine learning research as well as in many
industrial settings. This article reviews different techniques that can be used
for each of these three subtasks and discusses the main advantages and
disadvantages of each technique with references to theoretical and empirical
studies. Further, recommendations are given to encourage best yet feasible
practices in research and applications of machine learning. Common methods such
as the holdout method for model evaluation and selection are covered, which are
not recommended when working with small datasets. Different flavors of the
bootstrap technique are introduced for estimating the uncertainty of
performance estimates, as an alternative to confidence intervals via normal
approximation if bootstrapping is computationally feasible. Common
cross-validation techniques such as leave-one-out cross-validation and k-fold
cross-validation are reviewed, the bias-variance trade-off for choosing k is
discussed, and practical tips for the optimal choice of k are given based on
empirical evidence. Different statistical tests for algorithm comparisons are
presented, and strategies for dealing with multiple comparisons such as omnibus
tests and multiple-comparison corrections are discussed. Finally, alternative
methods for algorithm selection, such as the combined F-test 5x2
cross-validation and nested cross-validation, are recommended for comparing
machine learning algorithms when datasets are small.

24
The categories of this paper are: dg-ga
30
The categories of this paper are: q-alg
22
The categories of this paper are: chao-dyn
26
The categories of this paper are: bayes-an
10
The categories of this paper are: q-bio
33
The categories of this paper are: plasm-ph
7
The categories of this paper are: hep-th
31
The categories of this paper are: ao-sci
20
The categories of this paper are: acc-phys
3
The categories of this paper are: physics
25
The categories of this paper are: patt-sol
4
The categories of this paper are: cond-mat
6
The categories of this paper are: astro-ph
13
The categories of this paper are: quant-ph
18
The categories of this paper are: econ
17
The categories of this paper are: q-fin