Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Better Training using Weight-Constrained Stochastic Dynamics   We employ constraints to control the parameter space of deep neural networks
throughout training. The use of customized, appropriately designed constraints
can reduce the vanishing/exploding gradients problem, improve smoothness of
classification boundaries, control weight magnitudes and stabilize deep neural
networks, and thus enhance the robustness of training algorithms and the
generalization capabilities of neural networks. We provide a general approach
to efficiently incorporate constraints into a stochastic gradient Langevin
framework, allowing enhanced exploration of the loss landscape. We also present
specific examples of constrained training methods motivated by orthogonality
preservation for weight matrices and explicit weight normalizations.
Discretization schemes are provided both for the overdamped formulation of
Langevin dynamics and the underdamped form, in which momenta further improve
sampling efficiency. These optimization schemes can be used directly, without
needing to adapt neural network architecture design choices or to modify the
objective with regularization terms, and see performance improvements in
classification tasks.

31
The categories of this paper are: ao-sci
14
The categories of this paper are: math-ph
34
The categories of this paper are: chem-ph
10
The categories of this paper are: q-bio
30
The categories of this paper are: q-alg
3
The categories of this paper are: physics
13
The categories of this paper are: quant-ph
11
The categories of this paper are: hep-lat
28
The categories of this paper are: alg-geom
20
The categories of this paper are: acc-phys
24
The categories of this paper are: dg-ga
19
The categories of this paper are: eess
23
The categories of this paper are: solv-int
37
The categories of this paper are: supr-con
27
The categories of this paper are: comp-gas
26
The categories of this paper are: bayes-an