Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Towards Theoretical Analysis of Transformation Complexity of ReLU DNNs   This paper aims to theoretically analyze the complexity of feature
transformations encoded in DNNs with ReLU layers. We propose metrics to measure
three types of complexities of transformations based on the information theory.
We further discover and prove the strong correlation between the complexity and
the disentanglement of transformations. Based on the proposed metrics, we
analyze two typical phenomena of the change of the transformation complexity
during the training process, and explore the ceiling of a DNN's complexity. The
proposed metrics can also be used as a loss to learn a DNN with the minimum
complexity, which also controls the over-fitting level of the DNN and
influences adversarial robustness, adversarial transferability, and knowledge
consistency. Comprehensive comparative studies have provided new perspectives
to understand the DNN.

14
The categories of this paper are: math-ph
30
The categories of this paper are: q-alg
25
The categories of this paper are: patt-sol
28
The categories of this paper are: alg-geom
26
The categories of this paper are: bayes-an
32
The categories of this paper are: atom-ph
5
The categories of this paper are: gr-qc
11
The categories of this paper are: hep-lat
13
The categories of this paper are: quant-ph
29
The categories of this paper are: funct-an
16
The categories of this paper are: stat
23
The categories of this paper are: solv-int
4
The categories of this paper are: cond-mat
20
The categories of this paper are: acc-phys
19
The categories of this paper are: eess
9
The categories of this paper are: nlin