Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Some Theory For Practical Classifier Validation   We compare and contrast two approaches to validating a trained classifier
while using all in-sample data for training. One is simultaneous validation
over an organized set of hypotheses (SVOOSH), the well-known method that began
with VC theory. The other is withhold and gap (WAG). WAG withholds a validation
set, trains a holdout classifier on the remaining data, uses the validation
data to validate that classifier, then adds the rate of disagreement between
the holdout classifier and one trained using all in-sample data, which is an
upper bound on the difference in error rates. We show that complex hypothesis
classes and limited training data can make WAG a favorable alternative.

7
The categories of this paper are: hep-th
11
The categories of this paper are: hep-lat
31
The categories of this paper are: ao-sci
22
The categories of this paper are: chao-dyn
24
The categories of this paper are: dg-ga
28
The categories of this paper are: alg-geom
26
The categories of this paper are: bayes-an
5
The categories of this paper are: gr-qc
2
The categories of this paper are: math
3
The categories of this paper are: physics
8
The categories of this paper are: hep-ex
27
The categories of this paper are: comp-gas
36
The categories of this paper are: cmp-lg
0
The categories of this paper are: hep-ph
4
The categories of this paper are: cond-mat
21
The categories of this paper are: adap-org