Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Deep Networks from the Principle of Rate Reduction   This work attempts to interpret modern deep (convolutional) networks from the
principles of rate reduction and (shift) invariant classification. We show that
the basic iterative gradient ascent scheme for optimizing the rate reduction of
learned features naturally leads to a multi-layer deep network, one iteration
per layer. The layered architectures, linear and nonlinear operators, and even
parameters of the network are all explicitly constructed layer-by-layer in a
forward propagation fashion by emulating the gradient scheme. All components of
this "white box" network have precise optimization, statistical, and geometric
interpretation. This principled framework also reveals and justifies the role
of multi-channel lifting and sparse coding in early stage of deep networks.
Moreover, all linear operators of the so-derived network naturally become
multi-channel convolutions when we enforce classification to be rigorously
shift-invariant. The derivation also indicates that such a convolutional
network is significantly more efficient to construct and learn in the spectral
domain. Our preliminary simulations and experiments indicate that so
constructed deep network can already learn a good discriminative representation
even without any back propagation training.

14
The categories of this paper are: math-ph
5
The categories of this paper are: gr-qc
25
The categories of this paper are: patt-sol
3
The categories of this paper are: physics
10
The categories of this paper are: q-bio
24
The categories of this paper are: dg-ga
8
The categories of this paper are: hep-ex
12
The categories of this paper are: nucl-th
31
The categories of this paper are: ao-sci
19
The categories of this paper are: eess
15
The categories of this paper are: nucl-ex
33
The categories of this paper are: plasm-ph
37
The categories of this paper are: supr-con
9
The categories of this paper are: nlin
17
The categories of this paper are: q-fin
26
The categories of this paper are: bayes-an