Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Understanding, Categorizing and Predicting Semantic Image-Text Relations   Two modalities are often used to convey information in a complementary and
beneficial manner, e.g., in online news, videos, educational resources, or
scientific publications. The automatic understanding of semantic correlations
between text and associated images as well as their interplay has a great
potential for enhanced multimodal web search and recommender systems. However,
automatic understanding of multimodal information is still an unsolved research
problem. Recent approaches such as image captioning focus on precisely
describing visual content and translating it to text, but typically address
neither semantic interpretations nor the specific role or purpose of an
image-text constellation. In this paper, we go beyond previous work and
investigate, inspired by research in visual communication, useful semantic
image-text relations for multimodal information retrieval. We derive a
categorization of eight semantic image-text classes (e.g., "illustration" or
"anchorage") and show how they can systematically be characterized by a set of
three metrics: cross-modal mutual information, semantic correlation, and the
status relation of image and text. Furthermore, we present a deep learning
system to predict these classes by utilizing multimodal embeddings. To obtain a
sufficiently large amount of training data, we have automatically collected and
augmented data from a variety of data sets and web resources, which enables
future research on this topic. Experimental results on a demanding test set
demonstrate the feasibility of the approach.

18
The categories of this paper are: econ
20
The categories of this paper are: acc-phys
32
The categories of this paper are: atom-ph
34
The categories of this paper are: chem-ph
4
The categories of this paper are: cond-mat
12
The categories of this paper are: nucl-th
28
The categories of this paper are: alg-geom
11
The categories of this paper are: hep-lat
27
The categories of this paper are: comp-gas
24
The categories of this paper are: dg-ga
16
The categories of this paper are: stat
21
The categories of this paper are: adap-org
33
The categories of this paper are: plasm-ph
0
The categories of this paper are: hep-ph
17
The categories of this paper are: q-fin
2
The categories of this paper are: math