Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Generate, Annotate, and Learn: NLP with Synthetic Text   Semi-Supervised Learning (SSL) has seen success in many application domains,
but this success often hinges on the availability of task-specific unlabeled
data. Knowledge distillation (KD) has enabled effective optimization of compact
neural nets, achieving the best results when the knowledge of an expensive
network is distilled via fresh task-specific unlabeled data. However,
task-specific unlabeled data can be challenging to find, especially for NLP. We
investigate the use of generative models in synthesizing unlabeled data and
present a simple and general framework called "generate, annotate, and learn
(GAL)". A language model (LM) is used to synthesize in-domain unlabeled data.
Then, a classifier is used to annotate such data. Finally, synthetically
generated and annotated data is used to advance SSL, KD, and few-shot learning
on NLP and tabular tasks. To obtain a strong task-specific LM, we either
fine-tune a large LM on inputs from a specific task, or prompt a large LM with
a few input examples and conditionally generate more unlabeled examples. It
also yields a new state-of-the-art for 6-layer transformers on the GLUE
leaderboard. Finally, self-training with GAL offers large gains on four tabular
tasks from the UCI repository.

31
The categories of this paper are: ao-sci
19
The categories of this paper are: eess
4
The categories of this paper are: cond-mat
12
The categories of this paper are: nucl-th
37
The categories of this paper are: supr-con
2
The categories of this paper are: math
21
The categories of this paper are: adap-org
32
The categories of this paper are: atom-ph
26
The categories of this paper are: bayes-an
30
The categories of this paper are: q-alg
34
The categories of this paper are: chem-ph
0
The categories of this paper are: hep-ph
6
The categories of this paper are: astro-ph
24
The categories of this paper are: dg-ga
16
The categories of this paper are: stat
29
The categories of this paper are: funct-an