Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Linguistically-Informed Transformations (LIT): A Method for
  Automatically Generating Contrast Sets   Although large-scale pretrained language models, such as BERT and RoBERTa,
have achieved superhuman performance on in-distribution test sets, their
performance suffers on out-of-distribution test sets (e.g., on contrast sets).
Building contrast sets often re-quires human-expert annotation, which is
expensive and hard to create on a large scale. In this work, we propose a
Linguistically-Informed Transformation (LIT) method to automatically generate
contrast sets, which enables practitioners to explore linguistic phenomena of
interests as well as compose different phenomena. Experimenting with our method
on SNLI and MNLI shows that current pretrained language models, although being
claimed to contain sufficient linguistic knowledge, struggle on our
automatically generated contrast sets. Furthermore, we improve models'
performance on the contrast sets by apply-ing LIT to augment the training data,
without affecting performance on the original data.

5
The categories of this paper are: gr-qc
26
The categories of this paper are: bayes-an
33
The categories of this paper are: plasm-ph
17
The categories of this paper are: q-fin
14
The categories of this paper are: math-ph
22
The categories of this paper are: chao-dyn
25
The categories of this paper are: patt-sol
6
The categories of this paper are: astro-ph
36
The categories of this paper are: cmp-lg
28
The categories of this paper are: alg-geom
2
The categories of this paper are: math
21
The categories of this paper are: adap-org
13
The categories of this paper are: quant-ph
32
The categories of this paper are: atom-ph
16
The categories of this paper are: stat
29
The categories of this paper are: funct-an