Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Understanding Contrastive Learning Requires Incorporating Inductive
  Biases   Contrastive learning is a popular form of self-supervised learning that
encourages augmentations (views) of the same input to have more similar
representations compared to augmentations of different inputs. Recent attempts
to theoretically explain the success of contrastive learning on downstream
classification tasks prove guarantees depending on properties of {\em
augmentations} and the value of {\em contrastive loss} of representations. We
demonstrate that such analyses, that ignore {\em inductive biases} of the
function class and training algorithm, cannot adequately explain the success of
contrastive learning, even {\em provably} leading to vacuous guarantees in some
settings. Extensive experiments on image and text domains highlight the
ubiquity of this problem -- different function classes and algorithms behave
very differently on downstream tasks, despite having the same augmentations and
contrastive losses. Theoretical analysis is presented for the class of linear
representations, where incorporating inductive biases of the function class
allows contrastive learning to work with less stringent conditions compared to
prior analyses.

37
The categories of this paper are: supr-con
7
The categories of this paper are: hep-th
0
The categories of this paper are: hep-ph
27
The categories of this paper are: comp-gas
32
The categories of this paper are: atom-ph
25
The categories of this paper are: patt-sol
20
The categories of this paper are: acc-phys
11
The categories of this paper are: hep-lat
31
The categories of this paper are: ao-sci
19
The categories of this paper are: eess
24
The categories of this paper are: dg-ga
6
The categories of this paper are: astro-ph
12
The categories of this paper are: nucl-th
5
The categories of this paper are: gr-qc
13
The categories of this paper are: quant-ph
8
The categories of this paper are: hep-ex