Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Accounting for Agreement Phenomena in Sentence Comprehension with
  Transformer Language Models: Effects of Similarity-based Interference on
  Surprisal and Attention   We advance a novel explanation of similarity-based interference effects in
subject-verb and reflexive pronoun agreement processing, grounded in surprisal
values computed from a pretrained large-scale Transformer model, GPT-2.
Specifically, we show that surprisal of the verb or reflexive pronoun predicts
facilitatory interference effects in ungrammatical sentences, where a
distractor noun that matches in number with the verb or pronoun leads to faster
reading times, despite the distractor not participating in the agreement
relation. We review the human empirical evidence for such effects, including
recent meta-analyses and large-scale studies. We also show that attention
patterns (indexed by entropy and other measures) in the Transformer show
patterns of diffuse attention in the presence of similar distractors,
consistent with cue-based retrieval models of parsing. But in contrast to these
models, the attentional cues and memory representations are learned entirely
from the simple self-supervised task of predicting the next word.

7
The categories of this paper are: hep-th
14
The categories of this paper are: math-ph
25
The categories of this paper are: patt-sol
3
The categories of this paper are: physics
9
The categories of this paper are: nlin
31
The categories of this paper are: ao-sci
23
The categories of this paper are: solv-int
15
The categories of this paper are: nucl-ex
18
The categories of this paper are: econ
8
The categories of this paper are: hep-ex
13
The categories of this paper are: quant-ph
21
The categories of this paper are: adap-org
33
The categories of this paper are: plasm-ph
5
The categories of this paper are: gr-qc
0
The categories of this paper are: hep-ph
26
The categories of this paper are: bayes-an