Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Exploring Intra- and Inter-Video Relation for Surgical Semantic Scene
  Segmentation   Automatic surgical scene segmentation is fundamental for facilitating
cognitive intelligence in the modern operating theatre. Previous works rely on
conventional aggregation modules (e.g., dilated convolution, convolutional
LSTM), which only make use of the local context. In this paper, we propose a
novel framework STswinCL that explores the complementary intra- and inter-video
relations to boost segmentation performance, by progressively capturing the
global context. We firstly develop a hierarchy Transformer to capture
intra-video relation that includes richer spatial and temporal cues from
neighbor pixels and previous frames. A joint space-time window shift scheme is
proposed to efficiently aggregate these two cues into each pixel embedding.
Then, we explore inter-video relation via pixel-to-pixel contrastive learning,
which well structures the global embedding space. A multi-source contrast
training objective is developed to group the pixel embeddings across videos
with the ground-truth guidance, which is crucial for learning the global
property of the whole data. We extensively validate our approach on two public
surgical video benchmarks, including EndoVis18 Challenge and CaDIS dataset.
Experimental results demonstrate the promising performance of our method, which
consistently exceeds previous state-of-the-art approaches. Code will be
available at https://github.com/YuemingJin/STswinCL.

12
The categories of this paper are: nucl-th
37
The categories of this paper are: supr-con
0
The categories of this paper are: hep-ph
11
The categories of this paper are: hep-lat
23
The categories of this paper are: solv-int
28
The categories of this paper are: alg-geom
17
The categories of this paper are: q-fin
27
The categories of this paper are: comp-gas
19
The categories of this paper are: eess
3
The categories of this paper are: physics
13
The categories of this paper are: quant-ph
15
The categories of this paper are: nucl-ex
14
The categories of this paper are: math-ph
32
The categories of this paper are: atom-ph
5
The categories of this paper are: gr-qc
29
The categories of this paper are: funct-an