Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Information-Theoretic Bayes Risk Lower Bounds for Realizable Models   We derive information-theoretic lower bounds on the Bayes risk and
generalization error of realizable machine learning models. In particular, we
employ an analysis in which the rate-distortion function of the model
parameters bounds the required mutual information between the training samples
and the model parameters in order to learn a model up to a Bayes risk
constraint. For realizable models, we show that both the rate distortion
function and mutual information admit expressions that are convenient for
analysis. For models that are (roughly) lower Lipschitz in their parameters, we
bound the rate distortion function from below, whereas for VC classes, the
mutual information is bounded above by $d_\mathrm{vc}\log(n)$. When these
conditions match, the Bayes risk with respect to the zero-one loss scales no
faster than $\Omega(d_\mathrm{vc}/n)$, which matches known outer bounds and
minimax lower bounds up to logarithmic factors. We also consider the impact of
label noise, providing lower bounds when training and/or test samples are
corrupted.

19
The categories of this paper are: eess
15
The categories of this paper are: nucl-ex
6
The categories of this paper are: astro-ph
37
The categories of this paper are: supr-con
20
The categories of this paper are: acc-phys
21
The categories of this paper are: adap-org
28
The categories of this paper are: alg-geom
5
The categories of this paper are: gr-qc
26
The categories of this paper are: bayes-an
11
The categories of this paper are: hep-lat
32
The categories of this paper are: atom-ph
31
The categories of this paper are: ao-sci
30
The categories of this paper are: q-alg
9
The categories of this paper are: nlin
7
The categories of this paper are: hep-th
23
The categories of this paper are: solv-int