Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Transfer Learning Robustness in Multi-Class Categorization by
  Fine-Tuning Pre-Trained Contextualized Language Models   This study compares the effectiveness and robustness of multi-class
categorization of Amazon product data using transfer learning on pre-trained
contextualized language models. Specifically, we fine-tuned BERT and XLNet, two
bidirectional models that have achieved state-of-the-art performance on many
natural language tasks and benchmarks, including text classification. While
existing classification studies and benchmarks focus on binary targets, with
the exception of ordinal ranking tasks, here we examine the robustness of such
models as the number of classes grows from 1 to 20. Our experiments demonstrate
an approximately linear decrease in performance metrics (i.e., precision,
recall, $F_1$ score, and accuracy) with the number of class labels. BERT
consistently outperforms XLNet using identical hyperparameters on the entire
range of class label quantities for categorizing products based on their
textual descriptions. BERT is also more affordable than XLNet in terms of the
computational cost (i.e., time and memory) required for training. In all cases
studied, the performance degradation rates were estimated to be 1% per
additional class label.

5
The categories of this paper are: gr-qc
22
The categories of this paper are: chao-dyn
21
The categories of this paper are: adap-org
12
The categories of this paper are: nucl-th
15
The categories of this paper are: nucl-ex
0
The categories of this paper are: hep-ph
23
The categories of this paper are: solv-int
37
The categories of this paper are: supr-con
18
The categories of this paper are: econ
3
The categories of this paper are: physics
17
The categories of this paper are: q-fin
6
The categories of this paper are: astro-ph
32
The categories of this paper are: atom-ph
10
The categories of this paper are: q-bio
33
The categories of this paper are: plasm-ph
7
The categories of this paper are: hep-th