Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Cluster & Tune: Boost Cold Start Performance in Text Classification   In real-world scenarios, a text classification task often begins with a cold
start, when labeled data is scarce. In such cases, the common practice of
fine-tuning pre-trained models, such as BERT, for a target classification task,
is prone to produce poor performance. We suggest a method to boost the
performance of such models by adding an intermediate unsupervised
classification task, between the pre-training and fine-tuning phases. As such
an intermediate task, we perform clustering and train the pre-trained model on
predicting the cluster labels. We test this hypothesis on various data sets,
and show that this additional classification phase can significantly improve
performance, mainly for topical classification tasks, when the number of
labeled instances available for fine-tuning is only a couple of dozen to a few
hundred.

19
The categories of this paper are: eess
7
The categories of this paper are: hep-th
0
The categories of this paper are: hep-ph
28
The categories of this paper are: alg-geom
18
The categories of this paper are: econ
37
The categories of this paper are: supr-con
6
The categories of this paper are: astro-ph
30
The categories of this paper are: q-alg
10
The categories of this paper are: q-bio
23
The categories of this paper are: solv-int
24
The categories of this paper are: dg-ga
32
The categories of this paper are: atom-ph
35
The categories of this paper are: mtrl-th
11
The categories of this paper are: hep-lat
14
The categories of this paper are: math-ph
31
The categories of this paper are: ao-sci