Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: A Simple Baseline for Zero-shot Semantic Segmentation with Pre-trained
  Vision-language Model   Recently, zero-shot image classification by vision-language pre-training has
demonstrated incredible achievements, that the model can classify arbitrary
category without seeing additional annotated images of that category. However,
it is still unclear how to make the zero-shot recognition working well on
broader vision problems, such as object detection and semantic segmentation. In
this paper, we target for zero-shot semantic segmentation, by building it on an
off-the-shelf pre-trained vision-language model, i.e., CLIP. It is difficult
because semantic segmentation and the CLIP model perform on different visual
granularity, that semantic segmentation processes on pixels while CLIP performs
on images. To remedy the discrepancy on processing granularity, we refuse the
use of the prevalent one-stage FCN based framework, and advocate a two-stage
semantic segmentation framework, with the first stage extracting generalizable
mask proposals and the second stage leveraging an image based CLIP model to
perform zero-shot classification on the masked image crops which are generated
in the first stage. Our experimental results show that this simple framework
surpasses previous state-of-the-arts by a large margin: +29.5 hIoU on the
Pascal VOC 2012 dataset, and +8.9 hIoU on the COCO Stuff dataset. With its
simplicity and strong performance, we hope this framework to serve as a
baseline to facilitate the future research.

14
The categories of this paper are: math-ph
2
The categories of this paper are: math
29
The categories of this paper are: funct-an
20
The categories of this paper are: acc-phys
9
The categories of this paper are: nlin
23
The categories of this paper are: solv-int
25
The categories of this paper are: patt-sol
18
The categories of this paper are: econ
7
The categories of this paper are: hep-th
8
The categories of this paper are: hep-ex
15
The categories of this paper are: nucl-ex
0
The categories of this paper are: hep-ph
11
The categories of this paper are: hep-lat
35
The categories of this paper are: mtrl-th
17
The categories of this paper are: q-fin
10
The categories of this paper are: q-bio