Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural
  Network Representations Vary with Width and Depth   A key factor in the success of deep neural networks is the ability to scale
models to improve performance by varying the architecture depth and width. This
simple property of neural network design has resulted in highly effective
architectures for a variety of tasks. Nevertheless, there is limited
understanding of effects of depth and width on the learned representations. In
this paper, we study this fundamental question. We begin by investigating how
varying depth and width affects model hidden representations, finding a
characteristic block structure in the hidden representations of larger capacity
(wider or deeper) models. We demonstrate that this block structure arises when
model capacity is large relative to the size of the training set, and is
indicative of the underlying layers preserving and propagating the dominant
principal component of their representations. This discovery has important
ramifications for features learned by different models, namely, representations
outside the block structure are often similar across architectures with varying
widths and depths, but the block structure is unique to each model. We analyze
the output predictions of different model architectures, finding that even when
the overall accuracy is similar, wide and deep models exhibit distinctive error
patterns and variations across classes.

21
The categories of this paper are: adap-org
27
The categories of this paper are: comp-gas
6
The categories of this paper are: astro-ph
8
The categories of this paper are: hep-ex
0
The categories of this paper are: hep-ph
14
The categories of this paper are: math-ph
16
The categories of this paper are: stat
17
The categories of this paper are: q-fin
11
The categories of this paper are: hep-lat
36
The categories of this paper are: cmp-lg
28
The categories of this paper are: alg-geom
23
The categories of this paper are: solv-int
20
The categories of this paper are: acc-phys
7
The categories of this paper are: hep-th
22
The categories of this paper are: chao-dyn
19
The categories of this paper are: eess