Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Asymptotic convergence rate of Dropout on shallow linear neural networks   We analyze the convergence rate of gradient flows on objective functions
induced by Dropout and Dropconnect, when applying them to shallow linear Neural
Networks (NNs) - which can also be viewed as doing matrix factorization using a
particular regularizer. Dropout algorithms such as these are thus
regularization techniques that use 0,1-valued random variables to filter
weights during training in order to avoid coadaptation of features. By
leveraging a recent result on nonconvex optimization and conducting a careful
analysis of the set of minimizers as well as the Hessian of the loss function,
we are able to obtain (i) a local convergence proof of the gradient flow and
(ii) a bound on the convergence rate that depends on the data, the dropout
probability, and the width of the NN. Finally, we compare this theoretical
bound to numerical simulations, which are in qualitative agreement with the
convergence bound and match it when starting sufficiently close to a minimizer.

21
The categories of this paper are: adap-org
9
The categories of this paper are: nlin
36
The categories of this paper are: cmp-lg
14
The categories of this paper are: math-ph
20
The categories of this paper are: acc-phys
10
The categories of this paper are: q-bio
34
The categories of this paper are: chem-ph
30
The categories of this paper are: q-alg
18
The categories of this paper are: econ
23
The categories of this paper are: solv-int
15
The categories of this paper are: nucl-ex
25
The categories of this paper are: patt-sol
22
The categories of this paper are: chao-dyn
37
The categories of this paper are: supr-con
6
The categories of this paper are: astro-ph
8
The categories of this paper are: hep-ex