  With the advancement of generation models, AI-generated content (AIGC) is
becoming more realistic, flooding the Internet. A recent study suggests that
this phenomenon causes source bias in text retrieval for web search.
Specifically, neural retrieval models tend to rank generated texts higher than
human-written texts. In this paper, we extend the study of this bias to
cross-modal retrieval. Firstly, we successfully construct a suitable benchmark
to explore the existence of the bias. Subsequent extensive experiments on this
benchmark reveal that AI-generated images introduce an invisible relevance bias
to text-image retrieval models. Specifically, our experiments show that
text-image retrieval models tend to rank the AI-generated images higher than
the real images, even though the AI-generated images do not exhibit more
visually relevant features to the query than real images. This invisible
relevance bias is prevalent across retrieval models with varying training data
and architectures. Furthermore, our subsequent exploration reveals that the
inclusion of AI-generated images in the training data of the retrieval models
exacerbates the invisible relevance bias. The above phenomenon triggers a
vicious cycle, which makes the invisible relevance bias become more and more
serious. To elucidate the potential causes of invisible relevance and address
the aforementioned issues, we introduce an effective training method aimed at
alleviating the invisible relevance bias. Subsequently, we apply our proposed
debiasing method to retroactively identify the causes of invisible relevance,
revealing that the AI-generated images induce the image encoder to embed
additional information into their representation. This information exhibits a
certain consistency across generated images with different semantics and can
make the retriever estimate a higher relevance score.
