Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Deep Neural Networks Evolve Human-like Attention Distribution during
  Reading Comprehension   Attention is a key mechanism for information selection in both biological
brains and many state-of-the-art deep neural networks (DNNs). Here, we
investigate whether humans and DNNs allocate attention in comparable ways when
reading a text passage to subsequently answer a specific question. We analyze 3
transformer-based DNNs that reach human-level performance when trained to
perform the reading comprehension task. We find that the DNN attention
distribution quantitatively resembles human attention distribution measured by
fixation times. Human readers fixate longer on words that are more relevant to
the question-answering task, demonstrating that attention is modulated by
top-down reading goals, on top of lower-level visual and text features of the
stimulus. Further analyses reveal that the attention weights in DNNs are also
influenced by both top-down reading goals and lower-level stimulus features,
with the shallow layers more strongly influenced by lower-level text features
and the deep layers attending more to task-relevant words. Additionally, deep
layers' attention to task-relevant words gradually emerges when pre-trained DNN
models are fine-tuned to perform the reading comprehension task, which
coincides with the improvement in task performance. These results demonstrate
that DNNs can evolve human-like attention distribution through task
optimization, which suggests that human attention during goal-directed reading
comprehension is a consequence of task optimization.

29
The categories of this paper are: funct-an
4
The categories of this paper are: cond-mat
25
The categories of this paper are: patt-sol
24
The categories of this paper are: dg-ga
11
The categories of this paper are: hep-lat
23
The categories of this paper are: solv-int
8
The categories of this paper are: hep-ex
14
The categories of this paper are: math-ph
37
The categories of this paper are: supr-con
13
The categories of this paper are: quant-ph
32
The categories of this paper are: atom-ph
10
The categories of this paper are: q-bio
26
The categories of this paper are: bayes-an
20
The categories of this paper are: acc-phys
0
The categories of this paper are: hep-ph
28
The categories of this paper are: alg-geom