Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Relationship-based Neural Baby Talk   Understanding interactions between objects in an image is an important
element for generating captions. In this paper, we propose a relationship-based
neural baby talk (R-NBT) model to comprehensively investigate several types of
pairwise object interactions by encoding each image via three different
relationship-based graph attention networks (GATs). We study three main
relationships: \textit{spatial relationships} to explore geometric
interactions, \textit{semantic relationships} to extract semantic interactions,
and \textit{implicit relationships} to capture hidden information that could
not be modelled explicitly as above. We construct three relationship graphs
with the objects in an image as nodes, and the mutual relationships of pairwise
objects as edges. By exploring features of neighbouring regions individually
via GATs, we integrate different types of relationships into visual features of
each node. Experiments on COCO dataset show that our proposed R-NBT model
outperforms state-of-the-art models trained on COCO dataset in three image
caption generation tasks.

0
The categories of this paper are: hep-ph
2
The categories of this paper are: math
6
The categories of this paper are: astro-ph
35
The categories of this paper are: mtrl-th
25
The categories of this paper are: patt-sol
9
The categories of this paper are: nlin
13
The categories of this paper are: quant-ph
28
The categories of this paper are: alg-geom
4
The categories of this paper are: cond-mat
12
The categories of this paper are: nucl-th
27
The categories of this paper are: comp-gas
23
The categories of this paper are: solv-int
20
The categories of this paper are: acc-phys
31
The categories of this paper are: ao-sci
5
The categories of this paper are: gr-qc
18
The categories of this paper are: econ