Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Attention Is (not) All You Need for Commonsense Reasoning   The recently introduced BERT model exhibits strong performance on several
language understanding benchmarks. In this paper, we describe a simple
re-implementation of BERT for commonsense reasoning. We show that the
attentions produced by BERT can be directly utilized for tasks such as the
Pronoun Disambiguation Problem and Winograd Schema Challenge. Our proposed
attention-guided commonsense reasoning method is conceptually simple yet
empirically powerful. Experimental analysis on multiple datasets demonstrates
that our proposed system performs remarkably well on all cases while
outperforming the previously reported state of the art by a margin. While
results suggest that BERT seems to implicitly learn to establish complex
relationships between entities, solving commonsense reasoning tasks might
require more than unsupervised models learned from huge text corpora.

36
The categories of this paper are: cmp-lg
30
The categories of this paper are: q-alg
11
The categories of this paper are: hep-lat
17
The categories of this paper are: q-fin
10
The categories of this paper are: q-bio
33
The categories of this paper are: plasm-ph
6
The categories of this paper are: astro-ph
27
The categories of this paper are: comp-gas
8
The categories of this paper are: hep-ex
28
The categories of this paper are: alg-geom
20
The categories of this paper are: acc-phys
37
The categories of this paper are: supr-con
34
The categories of this paper are: chem-ph
3
The categories of this paper are: physics
21
The categories of this paper are: adap-org
9
The categories of this paper are: nlin