Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Visual Referring Expression Recognition: What Do Systems Actually Learn?   We present an empirical analysis of the state-of-the-art systems for
referring expression recognition -- the task of identifying the object in an
image referred to by a natural language expression -- with the goal of gaining
insight into how these systems reason about language and vision. Surprisingly,
we find strong evidence that even sophisticated and linguistically-motivated
models for this task may ignore the linguistic structure, instead relying on
shallow correlations introduced by unintended biases in the data selection and
annotation process. For example, we show that a system trained and tested on
the input image $\textit{without the input referring expression}$ can achieve a
precision of 71.2% in top-2 predictions. Furthermore, a system that predicts
only the object category given the input can achieve a precision of 84.2% in
top-2 predictions. These surprisingly positive results for what should be
deficient prediction scenarios suggest that careful analysis of what our models
are learning -- and further, how our data is constructed -- is critical as we
seek to make substantive progress on grounded language tasks.

37
The categories of this paper are: supr-con
9
The categories of this paper are: nlin
35
The categories of this paper are: mtrl-th
18
The categories of this paper are: econ
19
The categories of this paper are: eess
0
The categories of this paper are: hep-ph
25
The categories of this paper are: patt-sol
15
The categories of this paper are: nucl-ex
27
The categories of this paper are: comp-gas
34
The categories of this paper are: chem-ph
14
The categories of this paper are: math-ph
3
The categories of this paper are: physics
32
The categories of this paper are: atom-ph
31
The categories of this paper are: ao-sci
20
The categories of this paper are: acc-phys
12
The categories of this paper are: nucl-th