Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Efficient Contrastive Learning via Novel Data Augmentation and
  Curriculum Learning   We introduce EfficientCL, a memory-efficient continual pretraining method
that applies contrastive learning with novel data augmentation and curriculum
learning. For data augmentation, we stack two types of operation sequentially:
cutoff and PCA jittering. While pretraining steps proceed, we apply curriculum
learning by incrementing the augmentation degree for each difficulty step.
After data augmentation is finished, contrastive learning is applied on
projected embeddings of original and augmented examples. When finetuned on GLUE
benchmark, our model outperforms baseline models, especially for sentence-level
tasks. Additionally, this improvement is capable with only 70% of computational
memory compared to the baseline model.

16
The categories of this paper are: stat
25
The categories of this paper are: patt-sol
8
The categories of this paper are: hep-ex
18
The categories of this paper are: econ
12
The categories of this paper are: nucl-th
29
The categories of this paper are: funct-an
3
The categories of this paper are: physics
15
The categories of this paper are: nucl-ex
7
The categories of this paper are: hep-th
2
The categories of this paper are: math
4
The categories of this paper are: cond-mat
30
The categories of this paper are: q-alg
36
The categories of this paper are: cmp-lg
37
The categories of this paper are: supr-con
23
The categories of this paper are: solv-int
35
The categories of this paper are: mtrl-th