Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Enriched Pre-trained Transformers for Joint Slot Filling and Intent
  Detection   Detecting the user's intent and finding the corresponding slots among the
utterance's words are important tasks in natural language understanding. Their
interconnected nature makes their joint modeling a standard part of training
such models. Moreover, data scarceness and specialized vocabularies pose
additional challenges. Recently, the advances in pre-trained language models,
namely contextualized models such as ELMo and BERT have revolutionized the
field by tapping the potential of training very large models with just a few
steps of fine-tuning on a task-specific dataset. Here, we leverage such models,
namely BERT and RoBERTa, and we design a novel architecture on top of them.
Moreover, we propose an intent pooling attention mechanism, and we reinforce
the slot filling task by fusing intent distributions, word features, and token
representations. The experimental results on standard datasets show that our
model outperforms both the current non-BERT state of the art as well as some
stronger BERT-based baselines.

5
The categories of this paper are: gr-qc
24
The categories of this paper are: dg-ga
8
The categories of this paper are: hep-ex
25
The categories of this paper are: patt-sol
2
The categories of this paper are: math
28
The categories of this paper are: alg-geom
10
The categories of this paper are: q-bio
29
The categories of this paper are: funct-an
30
The categories of this paper are: q-alg
11
The categories of this paper are: hep-lat
27
The categories of this paper are: comp-gas
22
The categories of this paper are: chao-dyn
9
The categories of this paper are: nlin
4
The categories of this paper are: cond-mat
16
The categories of this paper are: stat
15
The categories of this paper are: nucl-ex