Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Resolution and Relevance Trade-offs in Deep Learning   Deep learning has been successfully applied to various tasks, but its
underlying mechanism remains unclear. Neural networks associate similar inputs
in the visible layer to the same state of hidden variables in deep layers. The
fraction of inputs that are associated to the same state is a natural measure
of similarity and is simply related to the cost in bits required to represent
these inputs. The degeneracy of states with the same information cost provides
instead a natural measure of noise and is simply related the entropy of the
frequency of states, that we call relevance. Representations with minimal
noise, at a given level of similarity (resolution), are those that maximise the
relevance. A signature of such efficient representations is that frequency
distributions follow power laws. We show, in extensive numerical experiments,
that deep neural networks extract a hierarchy of efficient representations from
data, because they i) achieve low levels of noise (i.e. high relevance) and ii)
exhibit power law distributions. We also find that the layer that is most
efficient to reliably generate patterns of training data is the one for which
relevance and resolution are traded at the same price, which implies that
frequency distribution follows Zipf's law.

12
The categories of this paper are: nucl-th
28
The categories of this paper are: alg-geom
8
The categories of this paper are: hep-ex
27
The categories of this paper are: comp-gas
18
The categories of this paper are: econ
14
The categories of this paper are: math-ph
15
The categories of this paper are: nucl-ex
6
The categories of this paper are: astro-ph
30
The categories of this paper are: q-alg
19
The categories of this paper are: eess
26
The categories of this paper are: bayes-an
4
The categories of this paper are: cond-mat
16
The categories of this paper are: stat
11
The categories of this paper are: hep-lat
37
The categories of this paper are: supr-con
2
The categories of this paper are: math