Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Towards Incremental Transformers: An Empirical Analysis of Transformer
  Models for Incremental NLU   Incremental processing allows interactive systems to respond based on partial
inputs, which is a desirable property e.g. in dialogue agents. The currently
popular Transformer architecture inherently processes sequences as a whole,
abstracting away the notion of time. Recent work attempts to apply Transformers
incrementally via restart-incrementality by repeatedly feeding, to an unchanged
model, increasingly longer input prefixes to produce partial outputs. However,
this approach is computationally costly and does not scale efficiently for long
sequences. In parallel, we witness efforts to make Transformers more efficient,
e.g. the Linear Transformer (LT) with a recurrence mechanism. In this work, we
examine the feasibility of LT for incremental NLU in English. Our results show
that the recurrent LT model has better incremental performance and faster
inference speed compared to the standard Transformer and LT with
restart-incrementality, at the cost of part of the non-incremental (full
sequence) quality. We show that the performance drop can be mitigated by
training the model to wait for right context before committing to an output and
that training with input prefixes is beneficial for delivering correct partial
outputs.

5
The categories of this paper are: gr-qc
33
The categories of this paper are: plasm-ph
28
The categories of this paper are: alg-geom
23
The categories of this paper are: solv-int
36
The categories of this paper are: cmp-lg
27
The categories of this paper are: comp-gas
31
The categories of this paper are: ao-sci
6
The categories of this paper are: astro-ph
34
The categories of this paper are: chem-ph
0
The categories of this paper are: hep-ph
17
The categories of this paper are: q-fin
18
The categories of this paper are: econ
12
The categories of this paper are: nucl-th
9
The categories of this paper are: nlin
19
The categories of this paper are: eess
26
The categories of this paper are: bayes-an