Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Demystify Optimization and Generalization of Over-parameterized
  PAC-Bayesian Learning   PAC-Bayesian is an analysis framework where the training error can be
expressed as the weighted average of the hypotheses in the posterior
distribution whilst incorporating the prior knowledge. In addition to being a
pure generalization bound analysis tool, PAC-Bayesian bound can also be
incorporated into an objective function to train a probabilistic neural
network, making them a powerful and relevant framework that can numerically
provide a tight generalization bound for supervised learning. For simplicity,
we call probabilistic neural network learned using training objectives derived
from PAC-Bayesian bounds as {\it PAC-Bayesian learning}. Despite their
empirical success, the theoretical analysis of PAC-Bayesian learning for neural
networks is rarely explored. This paper proposes a new class of convergence and
generalization analysis for PAC-Bayes learning when it is used to train the
over-parameterized neural networks by the gradient descent method. For a wide
probabilistic neural network, we show that when PAC-Bayes learning is applied,
the convergence result corresponds to solving a kernel ridge regression when
the probabilistic neural tangent kernel (PNTK) is used as its kernel. Based on
this finding, we further characterize the uniform PAC-Bayesian generalization
bound which improves over the Rademacher complexity-based bound for
non-probabilistic neural network. Finally, drawing the insight from our
theoretical results, we propose a proxy measure for efficient hyperparameters
selection, which is proven to be time-saving.

33
The categories of this paper are: plasm-ph
22
The categories of this paper are: chao-dyn
13
The categories of this paper are: quant-ph
0
The categories of this paper are: hep-ph
14
The categories of this paper are: math-ph
11
The categories of this paper are: hep-lat
15
The categories of this paper are: nucl-ex
5
The categories of this paper are: gr-qc
27
The categories of this paper are: comp-gas
10
The categories of this paper are: q-bio
24
The categories of this paper are: dg-ga
21
The categories of this paper are: adap-org
35
The categories of this paper are: mtrl-th
37
The categories of this paper are: supr-con
19
The categories of this paper are: eess
26
The categories of this paper are: bayes-an