Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: ZerO Initialization: Initializing Residual Networks with only Zeros and
  Ones   Deep neural networks are usually initialized with random weights, with
adequately selected initial variance to ensure stable signal propagation during
training. However, there is no consensus on how to select the variance, and
this becomes challenging especially as the number of layers grows. In this
work, we replace the widely used random weight initialization with a fully
deterministic initialization scheme ZerO, which initializes residual networks
with only zeros and ones. By augmenting the standard ResNet architectures with
a few extra skip connections and Hadamard transforms, ZerO allows us to start
the training from zeros and ones entirely. This has many benefits such as
improving reproducibility (by reducing the variance over different experimental
runs) and allowing network training without batch normalization. Surprisingly,
we find that ZerO achieves state-of-the-art performance over various image
classification datasets, including ImageNet, which suggests random weights may
be unnecessary for modern network initialization.

9
The categories of this paper are: nlin
33
The categories of this paper are: plasm-ph
37
The categories of this paper are: supr-con
4
The categories of this paper are: cond-mat
10
The categories of this paper are: q-bio
34
The categories of this paper are: chem-ph
31
The categories of this paper are: ao-sci
26
The categories of this paper are: bayes-an
24
The categories of this paper are: dg-ga
12
The categories of this paper are: nucl-th
21
The categories of this paper are: adap-org
22
The categories of this paper are: chao-dyn
27
The categories of this paper are: comp-gas
2
The categories of this paper are: math
18
The categories of this paper are: econ
16
The categories of this paper are: stat