Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Deep Learning and the Information Bottleneck Principle   Deep Neural Networks (DNNs) are analyzed via the theoretical framework of the
information bottleneck (IB) principle. We first show that any DNN can be
quantified by the mutual information between the layers and the input and
output variables. Using this representation we can calculate the optimal
information theoretic limits of the DNN and obtain finite sample generalization
bounds. The advantage of getting closer to the theoretical limit is
quantifiable both by the generalization bound and by the network's simplicity.
We argue that both the optimal architecture, number of layers and
features/connections at each layer, are related to the bifurcation points of
the information bottleneck tradeoff, namely, relevant compression of the input
layer with respect to the output layer. The hierarchical representations at the
layered network naturally correspond to the structural phase transitions along
the information curve. We believe that this new insight can lead to new
optimality bounds and deep learning algorithms.

5
The categories of this paper are: gr-qc
20
The categories of this paper are: acc-phys
23
The categories of this paper are: solv-int
16
The categories of this paper are: stat
8
The categories of this paper are: hep-ex
25
The categories of this paper are: patt-sol
34
The categories of this paper are: chem-ph
14
The categories of this paper are: math-ph
30
The categories of this paper are: q-alg
17
The categories of this paper are: q-fin
29
The categories of this paper are: funct-an
2
The categories of this paper are: math
31
The categories of this paper are: ao-sci
6
The categories of this paper are: astro-ph
7
The categories of this paper are: hep-th
12
The categories of this paper are: nucl-th