Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Legal Transformer Models May Not Always Help   Deep learning-based Natural Language Processing methods, especially
transformers, have achieved impressive performance in the last few years.
Applying those state-of-the-art NLP methods to legal activities to automate or
simplify some simple work is of great value. This work investigates the value
of domain adaptive pre-training and language adapters in legal NLP tasks. By
comparing the performance of language models with domain adaptive pre-training
on different tasks and different dataset splits, we show that domain adaptive
pre-training is only helpful with low-resource downstream tasks, thus far from
being a panacea. We also benchmark the performance of adapters in a typical
legal NLP task and show that they can yield similar performance to full model
tuning with much smaller training costs. As an additional result, we release
LegalRoBERTa, a RoBERTa model further pre-trained on legal corpora.

13
The categories of this paper are: quant-ph
17
The categories of this paper are: q-fin
8
The categories of this paper are: hep-ex
5
The categories of this paper are: gr-qc
16
The categories of this paper are: stat
11
The categories of this paper are: hep-lat
14
The categories of this paper are: math-ph
21
The categories of this paper are: adap-org
0
The categories of this paper are: hep-ph
37
The categories of this paper are: supr-con
36
The categories of this paper are: cmp-lg
27
The categories of this paper are: comp-gas
9
The categories of this paper are: nlin
7
The categories of this paper are: hep-th
12
The categories of this paper are: nucl-th
18
The categories of this paper are: econ