Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Segmentation-grounded Scene Graph Generation   Scene graph generation has emerged as an important problem in computer
vision. While scene graphs provide a grounded representation of objects, their
locations and relations in an image, they do so only at the granularity of
proposal bounding boxes. In this work, we propose the first, to our knowledge,
framework for pixel-level segmentation-grounded scene graph generation. Our
framework is agnostic to the underlying scene graph generation method and
address the lack of segmentation annotations in target scene graph datasets
(e.g., Visual Genome) through transfer and multi-task learning from, and with,
an auxiliary dataset (e.g., MS COCO). Specifically, each target object being
detected is endowed with a segmentation mask, which is expressed as a
lingual-similarity weighted linear combination over categories that have
annotations present in an auxiliary dataset. These inferred masks, along with a
novel Gaussian attention mechanism which grounds the relations at a pixel-level
within the image, allow for improved relation prediction. The entire framework
is end-to-end trainable and is learned in a multi-task manner with both target
and auxiliary datasets.

21
The categories of this paper are: adap-org
9
The categories of this paper are: nlin
6
The categories of this paper are: astro-ph
26
The categories of this paper are: bayes-an
20
The categories of this paper are: acc-phys
36
The categories of this paper are: cmp-lg
25
The categories of this paper are: patt-sol
8
The categories of this paper are: hep-ex
0
The categories of this paper are: hep-ph
5
The categories of this paper are: gr-qc
14
The categories of this paper are: math-ph
10
The categories of this paper are: q-bio
29
The categories of this paper are: funct-an
19
The categories of this paper are: eess
34
The categories of this paper are: chem-ph
28
The categories of this paper are: alg-geom