Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: ISD: Self-Supervised Learning by Iterative Similarity Distillation   Recently, contrastive learning has achieved great results in self-supervised
learning, where the main idea is to push two augmentations of an image
(positive pairs) closer compared to other random images (negative pairs). We
argue that not all random images are equal. Hence, we introduce a self
supervised learning algorithm where we use a soft similarity for the negative
images rather than a binary distinction between positive and negative pairs. We
iteratively distill a slowly evolving teacher model to the student model by
capturing the similarity of a query image to some random images and
transferring that knowledge to the student. We argue that our method is less
constrained compared to recent contrastive learning methods, so it can learn
better features. Specifically, our method should handle unbalanced and
unlabeled data better than existing contrastive learning methods, because the
randomly chosen negative set might include many samples that are semantically
similar to the query image. In this case, our method labels them as highly
similar while standard contrastive methods label them as negative pairs. Our
method achieves comparable results to the state-of-the-art models. We also show
that our method performs better in the settings where the unlabeled data is
unbalanced. Our code is available here: https://github.com/UMBCvision/ISD.

14
The categories of this paper are: math-ph
16
The categories of this paper are: stat
13
The categories of this paper are: quant-ph
11
The categories of this paper are: hep-lat
30
The categories of this paper are: q-alg
34
The categories of this paper are: chem-ph
23
The categories of this paper are: solv-int
12
The categories of this paper are: nucl-th
29
The categories of this paper are: funct-an
28
The categories of this paper are: alg-geom
37
The categories of this paper are: supr-con
27
The categories of this paper are: comp-gas
24
The categories of this paper are: dg-ga
6
The categories of this paper are: astro-ph
18
The categories of this paper are: econ
7
The categories of this paper are: hep-th