Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Visual Question: Predicting If a Crowd Will Agree on the Answer   Visual question answering (VQA) systems are emerging from a desire to empower
users to ask any natural language question about visual content and receive a
valid answer in response. However, close examination of the VQA problem reveals
an unavoidable, entangled problem that multiple humans may or may not always
agree on a single answer to a visual question. We train a model to
automatically predict from a visual question whether a crowd would agree on a
single answer. We then propose how to exploit this system in a novel
application to efficiently allocate human effort to collect answers to visual
questions. Specifically, we propose a crowdsourcing system that automatically
solicits fewer human responses when answer agreement is expected and more human
responses when answer disagreement is expected. Our system improves upon
existing crowdsourcing systems, typically eliminating at least 20% of human
effort with no loss to the information collected from the crowd.

29
The categories of this paper are: funct-an
37
The categories of this paper are: supr-con
12
The categories of this paper are: nucl-th
30
The categories of this paper are: q-alg
34
The categories of this paper are: chem-ph
15
The categories of this paper are: nucl-ex
33
The categories of this paper are: plasm-ph
8
The categories of this paper are: hep-ex
6
The categories of this paper are: astro-ph
17
The categories of this paper are: q-fin
25
The categories of this paper are: patt-sol
23
The categories of this paper are: solv-int
19
The categories of this paper are: eess
22
The categories of this paper are: chao-dyn
3
The categories of this paper are: physics
16
The categories of this paper are: stat