Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Convergence of End-to-End Training in Deep Unsupervised Contrastive
  Learning   Unsupervised contrastive learning has gained increasing attention in the
latest research and has proven to be a powerful method for learning
representations from unlabeled data. However, little theoretical analysis was
known for this framework. In this paper, we study the optimization of deep
unsupervised contrastive learning. We prove that, by applying end-to-end
training that simultaneously updates two deep over-parameterized neural
networks, one can find an approximate stationary solution for the non-convex
contrastive loss. This result is inherently different from the existing
over-parameterized analysis in the supervised setting because, in contrast to
learning a specific target function, unsupervised contrastive learning tries to
encode the unlabeled data distribution into the neural networks, which
generally has no optimal solution. Our analysis provides theoretical insights
into the practical success of these unsupervised pretraining methods.

0
The categories of this paper are: hep-ph
32
The categories of this paper are: atom-ph
6
The categories of this paper are: astro-ph
34
The categories of this paper are: chem-ph
24
The categories of this paper are: dg-ga
18
The categories of this paper are: econ
28
The categories of this paper are: alg-geom
13
The categories of this paper are: quant-ph
21
The categories of this paper are: adap-org
4
The categories of this paper are: cond-mat
22
The categories of this paper are: chao-dyn
5
The categories of this paper are: gr-qc
14
The categories of this paper are: math-ph
36
The categories of this paper are: cmp-lg
8
The categories of this paper are: hep-ex
33
The categories of this paper are: plasm-ph