Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Audio-Visual Scene-Aware Dialog   We introduce the task of scene-aware dialog. Our goal is to generate a
complete and natural response to a question about a scene, given video and
audio of the scene and the history of previous turns in the dialog. To answer
successfully, agents must ground concepts from the question in the video while
leveraging contextual cues from the dialog history. To benchmark this task, we
introduce the Audio Visual Scene-Aware Dialog (AVSD) Dataset. For each of more
than 11,000 videos of human actions from the Charades dataset, our dataset
contains a dialog about the video, plus a final summary of the video by one of
the dialog participants. We train several baseline systems for this task and
evaluate the performance of the trained models using both qualitative and
quantitative metrics. Our results indicate that models must utilize all the
available inputs (video, audio, question, and dialog history) to perform best
on this dataset.

32
The categories of this paper are: atom-ph
8
The categories of this paper are: hep-ex
29
The categories of this paper are: funct-an
14
The categories of this paper are: math-ph
17
The categories of this paper are: q-fin
22
The categories of this paper are: chao-dyn
16
The categories of this paper are: stat
18
The categories of this paper are: econ
35
The categories of this paper are: mtrl-th
11
The categories of this paper are: hep-lat
26
The categories of this paper are: bayes-an
13
The categories of this paper are: quant-ph
34
The categories of this paper are: chem-ph
3
The categories of this paper are: physics
7
The categories of this paper are: hep-th
21
The categories of this paper are: adap-org