Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Convergence of a Relaxed Variable Splitting Method for Learning Sparse
  Neural Networks via $\ell_1, \ell_0$, and transformed-$\ell_1$ Penalties   Sparsification of neural networks is one of the effective complexity
reduction methods to improve efficiency and generalizability. We consider the
problem of learning a one hidden layer convolutional neural network with ReLU
activation function via gradient descent under sparsity promoting penalties. It
is known that when the input data is Gaussian distributed, no-overlap networks
(without penalties) in regression problems with ground truth can be learned in
polynomial time at high probability. We propose a relaxed variable splitting
method integrating thresholding and gradient descent to overcome the lack of
non-smoothness in the loss function. The sparsity in network weight is realized
during the optimization (training) process. We prove that under $\ell_1,
\ell_0$; and transformed-$\ell_1$ penalties, no-overlap networks can be learned
with high probability, and the iterative weights converge to a global limit
which is a transformation of the true weight under a novel thresholding
operation. Numerical experiments confirm theoretical findings, and compare the
accuracy and sparsity trade-off among the penalties.

20
The categories of this paper are: acc-phys
7
The categories of this paper are: hep-th
10
The categories of this paper are: q-bio
34
The categories of this paper are: chem-ph
4
The categories of this paper are: cond-mat
33
The categories of this paper are: plasm-ph
11
The categories of this paper are: hep-lat
31
The categories of this paper are: ao-sci
0
The categories of this paper are: hep-ph
3
The categories of this paper are: physics
17
The categories of this paper are: q-fin
16
The categories of this paper are: stat
12
The categories of this paper are: nucl-th
5
The categories of this paper are: gr-qc
23
The categories of this paper are: solv-int
35
The categories of this paper are: mtrl-th