Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Characterizing signal propagation to close the performance gap in
  unnormalized ResNets   Batch Normalization is a key component in almost all state-of-the-art image
classifiers, but it also introduces practical challenges: it breaks the
independence between training examples within a batch, can incur compute and
memory overhead, and often results in unexpected bugs. Building on recent
theoretical analyses of deep ResNets at initialization, we propose a simple set
of analysis tools to characterize signal propagation on the forward pass, and
leverage these tools to design highly performant ResNets without activation
normalization layers. Crucial to our success is an adapted version of the
recently proposed Weight Standardization. Our analysis tools show how this
technique preserves the signal in networks with ReLU or Swish activation
functions by ensuring that the per-channel activation means do not grow with
depth. Across a range of FLOP budgets, our networks attain performance
competitive with the state-of-the-art EfficientNets on ImageNet.

18
The categories of this paper are: econ
37
The categories of this paper are: supr-con
31
The categories of this paper are: ao-sci
4
The categories of this paper are: cond-mat
12
The categories of this paper are: nucl-th
30
The categories of this paper are: q-alg
25
The categories of this paper are: patt-sol
9
The categories of this paper are: nlin
24
The categories of this paper are: dg-ga
22
The categories of this paper are: chao-dyn
36
The categories of this paper are: cmp-lg
23
The categories of this paper are: solv-int
7
The categories of this paper are: hep-th
32
The categories of this paper are: atom-ph
17
The categories of this paper are: q-fin
21
The categories of this paper are: adap-org