Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Langevin Dynamics with Continuous Tempering for Training Deep Neural
  Networks   Minimizing non-convex and high-dimensional objective functions is
challenging, especially when training modern deep neural networks. In this
paper, a novel approach is proposed which divides the training process into two
consecutive phases to obtain better generalization performance: Bayesian
sampling and stochastic optimization. The first phase is to explore the energy
landscape and to capture the "fat" modes; and the second one is to fine-tune
the parameter learned from the first phase. In the Bayesian learning phase, we
apply continuous tempering and stochastic approximation into the Langevin
dynamics to create an efficient and effective sampler, in which the temperature
is adjusted automatically according to the designed "temperature dynamics".
These strategies can overcome the challenge of early trapping into bad local
minima and have achieved remarkable improvements in various types of neural
networks as shown in our theoretical analysis and empirical experiments.

20
The categories of this paper are: acc-phys
12
The categories of this paper are: nucl-th
22
The categories of this paper are: chao-dyn
6
The categories of this paper are: astro-ph
3
The categories of this paper are: physics
7
The categories of this paper are: hep-th
5
The categories of this paper are: gr-qc
29
The categories of this paper are: funct-an
19
The categories of this paper are: eess
36
The categories of this paper are: cmp-lg
2
The categories of this paper are: math
0
The categories of this paper are: hep-ph
21
The categories of this paper are: adap-org
15
The categories of this paper are: nucl-ex
8
The categories of this paper are: hep-ex
37
The categories of this paper are: supr-con