Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: DRILL: Dynamic Representations for Imbalanced Lifelong Learning   Continual or lifelong learning has been a long-standing challenge in machine
learning to date, especially in natural language processing (NLP). Although
state-of-the-art language models such as BERT have ushered in a new era in this
field due to their outstanding performance in multitask learning scenarios,
they suffer from forgetting when being exposed to a continuous stream of data
with shifting data distributions. In this paper, we introduce DRILL, a novel
continual learning architecture for open-domain text classification. DRILL
leverages a biologically inspired self-organizing neural architecture to
selectively gate latent language representations from BERT in a
task-incremental manner. We demonstrate in our experiments that DRILL
outperforms current methods in a realistic scenario of imbalanced,
non-stationary data without prior knowledge about task boundaries. To the best
of our knowledge, DRILL is the first of its kind to use a self-organizing
neural architecture for open-domain lifelong learning in NLP.

8
The categories of this paper are: hep-ex
4
The categories of this paper are: cond-mat
25
The categories of this paper are: patt-sol
16
The categories of this paper are: stat
6
The categories of this paper are: astro-ph
24
The categories of this paper are: dg-ga
33
The categories of this paper are: plasm-ph
10
The categories of this paper are: q-bio
3
The categories of this paper are: physics
9
The categories of this paper are: nlin
11
The categories of this paper are: hep-lat
22
The categories of this paper are: chao-dyn
37
The categories of this paper are: supr-con
15
The categories of this paper are: nucl-ex
36
The categories of this paper are: cmp-lg
5
The categories of this paper are: gr-qc