Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Unified Image and Video Saliency Modeling   Visual saliency modeling for images and videos is treated as two independent
tasks in recent computer vision literature. While image saliency modeling is a
well-studied problem and progress on benchmarks like SALICON and MIT300 is
slowing, video saliency models have shown rapid gains on the recent DHF1K
benchmark. Here, we take a step back and ask: Can image and video saliency
modeling be approached via a unified model, with mutual benefit? We identify
different sources of domain shift between image and video saliency data and
between different video saliency datasets as a key challenge for effective
joint modelling. To address this we propose four novel domain adaptation
techniques - Domain-Adaptive Priors, Domain-Adaptive Fusion, Domain-Adaptive
Smoothing and Bypass-RNN - in addition to an improved formulation of learned
Gaussian priors. We integrate these techniques into a simple and lightweight
encoder-RNN-decoder-style network, UNISAL, and train it jointly with image and
video saliency data. We evaluate our method on the video saliency datasets
DHF1K, Hollywood-2 and UCF-Sports, and the image saliency datasets SALICON and
MIT300. With one set of parameters, UNISAL achieves state-of-the-art
performance on all video saliency datasets and is on par with the
state-of-the-art for image saliency datasets, despite faster runtime and a 5 to
20-fold smaller model size compared to all competing deep methods. We provide
retrospective analyses and ablation studies which confirm the importance of the
domain shift modeling. The code is available at
https://github.com/rdroste/unisal

16
The categories of this paper are: stat
3
The categories of this paper are: physics
20
The categories of this paper are: acc-phys
37
The categories of this paper are: supr-con
9
The categories of this paper are: nlin
33
The categories of this paper are: plasm-ph
25
The categories of this paper are: patt-sol
2
The categories of this paper are: math
32
The categories of this paper are: atom-ph
30
The categories of this paper are: q-alg
12
The categories of this paper are: nucl-th
0
The categories of this paper are: hep-ph
35
The categories of this paper are: mtrl-th
8
The categories of this paper are: hep-ex
36
The categories of this paper are: cmp-lg
23
The categories of this paper are: solv-int