Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Do Data-based Curricula Work?   Current state-of-the-art NLP systems use large neural networks that require
lots of computational resources for training. Inspired by human knowledge
acquisition, researchers have proposed curriculum learning, - sequencing of
tasks (task-based curricula) or ordering and sampling of the datasets
(data-based curricula) that facilitate training. This work investigates the
benefits of data-based curriculum learning for large modern language models
such as BERT and T5. We experiment with various curricula based on a range of
complexity measures and different sampling strategies. Extensive experiments on
different NLP tasks show that curricula based on various complexity measures
rarely has any benefits while random sampling performs either as well or better
than curricula.

8
The categories of this paper are: hep-ex
11
The categories of this paper are: hep-lat
3
The categories of this paper are: physics
35
The categories of this paper are: mtrl-th
14
The categories of this paper are: math-ph
18
The categories of this paper are: econ
13
The categories of this paper are: quant-ph
26
The categories of this paper are: bayes-an
16
The categories of this paper are: stat
24
The categories of this paper are: dg-ga
12
The categories of this paper are: nucl-th
36
The categories of this paper are: cmp-lg
17
The categories of this paper are: q-fin
25
The categories of this paper are: patt-sol
23
The categories of this paper are: solv-int
33
The categories of this paper are: plasm-ph