Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Data-Efficient Language-Supervised Zero-Shot Learning with
  Self-Distillation   Traditional computer vision models are trained to predict a fixed set of
predefined categories. Recently, natural language has been shown to be a
broader and richer source of supervision that provides finer descriptions to
visual concepts than supervised "gold" labels. Previous works, such as CLIP,
use a simple pretraining task of predicting the pairings between images and
text captions. CLIP, however, is data hungry and requires more than 400M image
text pairs for training. We propose a data-efficient contrastive distillation
method that uses soft labels to learn from noisy image-text pairs. Our model
transfers knowledge from pretrained image and sentence encoders and achieves
strong performance with only 3M image text pairs, 133x smaller than CLIP. Our
method exceeds the previous SoTA of general zero-shot learning on ImageNet
21k+1k by 73% relatively with a ResNet50 image encoder and DeCLUTR text
encoder. We also beat CLIP by 10.5% relatively on zero-shot evaluation on
Google Open Images (19,958 classes).

25
The categories of this paper are: patt-sol
6
The categories of this paper are: astro-ph
13
The categories of this paper are: quant-ph
28
The categories of this paper are: alg-geom
34
The categories of this paper are: chem-ph
21
The categories of this paper are: adap-org
2
The categories of this paper are: math
32
The categories of this paper are: atom-ph
36
The categories of this paper are: cmp-lg
14
The categories of this paper are: math-ph
5
The categories of this paper are: gr-qc
35
The categories of this paper are: mtrl-th
22
The categories of this paper are: chao-dyn
26
The categories of this paper are: bayes-an
18
The categories of this paper are: econ
9
The categories of this paper are: nlin