Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Supervision Accelerates Pre-training in Contrastive Semi-Supervised
  Learning of Visual Representations   We investigate a strategy for improving the efficiency of contrastive
learning of visual representations by leveraging a small amount of supervised
information during pre-training. We propose a semi-supervised loss, SuNCEt,
based on noise-contrastive estimation and neighbourhood component analysis,
that aims to distinguish examples of different classes in addition to the
self-supervised instance-wise pretext tasks. On ImageNet, we find that SuNCEt
can be used to match the semi-supervised learning accuracy of previous
contrastive approaches while using less than half the amount of pre-training
and compute. Our main insight is that leveraging even a small amount of labeled
data during pre-training, and not only during fine-tuning, provides an
important signal that can significantly accelerate contrastive learning of
visual representations. Our code is available online at
github.com/facebookresearch/suncet.

33
The categories of this paper are: plasm-ph
6
The categories of this paper are: astro-ph
10
The categories of this paper are: q-bio
20
The categories of this paper are: acc-phys
9
The categories of this paper are: nlin
8
The categories of this paper are: hep-ex
36
The categories of this paper are: cmp-lg
24
The categories of this paper are: dg-ga
12
The categories of this paper are: nucl-th
7
The categories of this paper are: hep-th
27
The categories of this paper are: comp-gas
11
The categories of this paper are: hep-lat
26
The categories of this paper are: bayes-an
5
The categories of this paper are: gr-qc
0
The categories of this paper are: hep-ph
19
The categories of this paper are: eess