Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Visuo-Linguistic Question Answering (VLQA) Challenge   Understanding images and text together is an important aspect of cognition
and building advanced Artificial Intelligence (AI) systems. As a community, we
have achieved good benchmarks over language and vision domains separately,
however joint reasoning is still a challenge for state-of-the-art computer
vision and natural language processing (NLP) systems. We propose a novel task
to derive joint inference about a given image-text modality and compile the
Visuo-Linguistic Question Answering (VLQA) challenge corpus in a question
answering setting. Each dataset item consists of an image and a reading
passage, where questions are designed to combine both visual and textual
information i.e., ignoring either modality would make the question
unanswerable. We first explore the best existing vision-language architectures
to solve VLQA subsets and show that they are unable to reason well. We then
develop a modular method with slightly better baseline performance, but it is
still far behind human performance. We believe that VLQA will be a good
benchmark for reasoning over a visuo-linguistic context. The dataset, code and
leaderboard is available at https://shailaja183.github.io/vlqa/.

5
The categories of this paper are: gr-qc
33
The categories of this paper are: plasm-ph
34
The categories of this paper are: chem-ph
19
The categories of this paper are: eess
27
The categories of this paper are: comp-gas
25
The categories of this paper are: patt-sol
8
The categories of this paper are: hep-ex
14
The categories of this paper are: math-ph
12
The categories of this paper are: nucl-th
13
The categories of this paper are: quant-ph
21
The categories of this paper are: adap-org
22
The categories of this paper are: chao-dyn
28
The categories of this paper are: alg-geom
35
The categories of this paper are: mtrl-th
31
The categories of this paper are: ao-sci
30
The categories of this paper are: q-alg