Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: The World of an Octopus: How Reporting Bias Influences a Language
  Model's Perception of Color   Recent work has raised concerns about the inherent limitations of text-only
pretraining. In this paper, we first demonstrate that reporting bias, the
tendency of people to not state the obvious, is one of the causes of this
limitation, and then investigate to what extent multimodal training can
mitigate this issue. To accomplish this, we 1) generate the Color Dataset
(CoDa), a dataset of human-perceived color distributions for 521 common
objects; 2) use CoDa to analyze and compare the color distribution found in
text, the distribution captured by language models, and a human's perception of
color; and 3) investigate the performance differences between text-only and
multimodal models on CoDa. Our results show that the distribution of colors
that a language model recovers correlates more strongly with the inaccurate
distribution found in text than with the ground-truth, supporting the claim
that reporting bias negatively impacts and inherently limits text-only
training. We then demonstrate that multimodal models can leverage their visual
training to mitigate these effects, providing a promising avenue for future
research.

13
The categories of this paper are: quant-ph
24
The categories of this paper are: dg-ga
20
The categories of this paper are: acc-phys
14
The categories of this paper are: math-ph
18
The categories of this paper are: econ
30
The categories of this paper are: q-alg
23
The categories of this paper are: solv-int
26
The categories of this paper are: bayes-an
27
The categories of this paper are: comp-gas
21
The categories of this paper are: adap-org
29
The categories of this paper are: funct-an
37
The categories of this paper are: supr-con
0
The categories of this paper are: hep-ph
4
The categories of this paper are: cond-mat
16
The categories of this paper are: stat
34
The categories of this paper are: chem-ph