Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Representation Learning on Visual-Symbolic Graphs for Video
  Understanding   Events in natural videos typically arise from spatio-temporal interactions
between actors and objects and involve multiple co-occurring activities and
object classes. To capture this rich visual and semantic context, we propose
using two graphs: (1) an attributed spatio-temporal visual graph whose nodes
correspond to actors and objects and whose edges encode different types of
interactions, and (2) a symbolic graph that models semantic relationships. We
further propose a graph neural network for refining the representations of
actors, objects and their interactions on the resulting hybrid graph. Our model
goes beyond current approaches that assume nodes and edges are of the same
type, operate on graphs with fixed edge weights and do not use a symbolic
graph. In particular, our framework: a) has specialized attention-based message
functions for different node and edge types; b) uses visual edge features; c)
integrates visual evidence with label relationships; and d) performs global
reasoning in the semantic space. Experiments on challenging video understanding
tasks, such as temporal action localization on the Charades dataset, show that
the proposed method leads to state-of-the-art performance.

15
The categories of this paper are: nucl-ex
11
The categories of this paper are: hep-lat
9
The categories of this paper are: nlin
10
The categories of this paper are: q-bio
21
The categories of this paper are: adap-org
37
The categories of this paper are: supr-con
0
The categories of this paper are: hep-ph
17
The categories of this paper are: q-fin
31
The categories of this paper are: ao-sci
2
The categories of this paper are: math
6
The categories of this paper are: astro-ph
22
The categories of this paper are: chao-dyn
23
The categories of this paper are: solv-int
8
The categories of this paper are: hep-ex
30
The categories of this paper are: q-alg
25
The categories of this paper are: patt-sol