Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Can Open Domain Question Answering Systems Answer Visual Knowledge
  Questions?   The task of Outside Knowledge Visual Question Answering (OKVQA) requires an
automatic system to answer natural language questions about pictures and images
using external knowledge. We observe that many visual questions, which contain
deictic referential phrases referring to entities in the image, can be
rewritten as "non-grounded" questions and can be answered by existing
text-based question answering systems. This allows for the reuse of existing
text-based Open Domain Question Answering (QA) Systems for visual question
answering. In this work, we propose a potentially data-efficient approach that
reuses existing systems for (a) image analysis, (b) question rewriting, and (c)
text-based question answering to answer such visual questions. Given an image
and a question pertaining to that image (a visual question), we first extract
the entities present in the image using pre-trained object and scene
classifiers. Using these detected entities, the visual questions can be
rewritten so as to be answerable by open domain QA systems. We explore two
rewriting strategies: (1) an unsupervised method using BERT for masking and
rewriting, and (2) a weakly supervised approach that combines adaptive
rewriting and reinforcement learning techniques to use the implicit feedback
from the QA system. We test our strategies on the publicly available OKVQA
dataset and obtain a competitive performance with state-of-the-art models while
using only 10% of the training data.

21
The categories of this paper are: adap-org
36
The categories of this paper are: cmp-lg
4
The categories of this paper are: cond-mat
10
The categories of this paper are: q-bio
13
The categories of this paper are: quant-ph
12
The categories of this paper are: nucl-th
14
The categories of this paper are: math-ph
31
The categories of this paper are: ao-sci
35
The categories of this paper are: mtrl-th
25
The categories of this paper are: patt-sol
15
The categories of this paper are: nucl-ex
5
The categories of this paper are: gr-qc
22
The categories of this paper are: chao-dyn
30
The categories of this paper are: q-alg
33
The categories of this paper are: plasm-ph
16
The categories of this paper are: stat