Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: On the Regularization Properties of Structured Dropout   Dropout and its extensions (eg. DropBlock and DropConnect) are popular
heuristics for training neural networks, which have been shown to improve
generalization performance in practice. However, a theoretical understanding of
their optimization and regularization properties remains elusive. Recent work
shows that in the case of single hidden-layer linear networks, Dropout is a
stochastic gradient descent method for minimizing a regularized loss, and that
the regularizer induces solutions that are low-rank and balanced. In this work
we show that for single hidden-layer linear networks, DropBlock induces
spectral k-support norm regularization, and promotes solutions that are
low-rank and have factors with equal norm. We also show that the global
minimizer for DropBlock can be computed in closed form, and that DropConnect is
equivalent to Dropout. We then show that some of these results can be extended
to a general class of Dropout-strategies, and, with some assumptions, to deep
non-linear networks when Dropout is applied to the last layer. We verify our
theoretical claims and assumptions experimentally with commonly used network
architectures.

21
The categories of this paper are: adap-org
37
The categories of this paper are: supr-con
30
The categories of this paper are: q-alg
5
The categories of this paper are: gr-qc
24
The categories of this paper are: dg-ga
0
The categories of this paper are: hep-ph
11
The categories of this paper are: hep-lat
14
The categories of this paper are: math-ph
15
The categories of this paper are: nucl-ex
8
The categories of this paper are: hep-ex
23
The categories of this paper are: solv-int
28
The categories of this paper are: alg-geom
25
The categories of this paper are: patt-sol
31
The categories of this paper are: ao-sci
3
The categories of this paper are: physics
34
The categories of this paper are: chem-ph