Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: ScatSimCLR: self-supervised contrastive learning with pretext task
  regularization for small-scale datasets   In this paper, we consider a problem of self-supervised learning for
small-scale datasets based on contrastive loss between multiple views of the
data, which demonstrates the state-of-the-art performance in classification
task. Despite the reported results, such factors as the complexity of training
requiring complex architectures, the needed number of views produced by data
augmentation, and their impact on the classification accuracy are understudied
problems. To establish the role of these factors, we consider an architecture
of contrastive loss system such as SimCLR, where baseline model is replaced by
geometrically invariant "hand-crafted" network ScatNet with small trainable
adapter network and argue that the number of parameters of the whole system and
the number of views can be considerably reduced while practically preserving
the same classification accuracy. In addition, we investigate the impact of
regularization strategies using pretext task learning based on an estimation of
parameters of augmentation transform such as rotation and jigsaw permutation
for both traditional baseline models and ScatNet based models. Finally, we
demonstrate that the proposed architecture with pretext task learning
regularization achieves the state-of-the-art classification performance with a
smaller number of trainable parameters and with reduced number of views.

34
The categories of this paper are: chem-ph
5
The categories of this paper are: gr-qc
22
The categories of this paper are: chao-dyn
29
The categories of this paper are: funct-an
12
The categories of this paper are: nucl-th
30
The categories of this paper are: q-alg
24
The categories of this paper are: dg-ga
36
The categories of this paper are: cmp-lg
9
The categories of this paper are: nlin
0
The categories of this paper are: hep-ph
19
The categories of this paper are: eess
20
The categories of this paper are: acc-phys
23
The categories of this paper are: solv-int
17
The categories of this paper are: q-fin
3
The categories of this paper are: physics
27
The categories of this paper are: comp-gas