Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Data-dependent Initializations of Convolutional Neural Networks   Convolutional Neural Networks spread through computer vision like a wildfire,
impacting almost all visual tasks imaginable. Despite this, few researchers
dare to train their models from scratch. Most work builds on one of a handful
of ImageNet pre-trained models, and fine-tunes or adapts these for specific
tasks. This is in large part due to the difficulty of properly initializing
these networks from scratch. A small miscalibration of the initial weights
leads to vanishing or exploding gradients, as well as poor convergence
properties. In this work we present a fast and simple data-dependent
initialization procedure, that sets the weights of a network such that all
units in the network train at roughly the same rate, avoiding vanishing or
exploding gradients. Our initialization matches the current state-of-the-art
unsupervised or self-supervised pre-training methods on standard computer
vision tasks, such as image classification and object detection, while being
roughly three orders of magnitude faster. When combined with pre-training
methods, our initialization significantly outperforms prior work, narrowing the
gap between supervised and unsupervised pre-training.

11
The categories of this paper are: hep-lat
20
The categories of this paper are: acc-phys
9
The categories of this paper are: nlin
34
The categories of this paper are: chem-ph
0
The categories of this paper are: hep-ph
32
The categories of this paper are: atom-ph
8
The categories of this paper are: hep-ex
36
The categories of this paper are: cmp-lg
25
The categories of this paper are: patt-sol
21
The categories of this paper are: adap-org
35
The categories of this paper are: mtrl-th
5
The categories of this paper are: gr-qc
30
The categories of this paper are: q-alg
19
The categories of this paper are: eess
12
The categories of this paper are: nucl-th
31
The categories of this paper are: ao-sci