Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Learning to Relate from Captions and Bounding Boxes   In this work, we propose a novel approach that predicts the relationships
between various entities in an image in a weakly supervised manner by relying
on image captions and object bounding box annotations as the sole source of
supervision. Our proposed approach uses a top-down attention mechanism to align
entities in captions to objects in the image, and then leverage the syntactic
structure of the captions to align the relations. We use these alignments to
train a relation classification network, thereby obtaining both grounded
captions and dense relationships. We demonstrate the effectiveness of our model
on the Visual Genome dataset by achieving a recall@50 of 15% and recall@100 of
25% on the relationships present in the image. We also show that the model
successfully predicts relations that are not present in the corresponding
captions.

36
The categories of this paper are: cmp-lg
15
The categories of this paper are: nucl-ex
3
The categories of this paper are: physics
31
The categories of this paper are: ao-sci
26
The categories of this paper are: bayes-an
28
The categories of this paper are: alg-geom
27
The categories of this paper are: comp-gas
14
The categories of this paper are: math-ph
33
The categories of this paper are: plasm-ph
23
The categories of this paper are: solv-int
19
The categories of this paper are: eess
30
The categories of this paper are: q-alg
0
The categories of this paper are: hep-ph
22
The categories of this paper are: chao-dyn
25
The categories of this paper are: patt-sol
12
The categories of this paper are: nucl-th