Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: iBOT: Image BERT Pre-Training with Online Tokenizer   The success of language Transformers is primarily attributed to the pretext
task of masked language modeling (MLM), where texts are first tokenized into
semantically meaningful pieces. In this work, we study masked image modeling
(MIM) and indicate the advantages and challenges of using a semantically
meaningful visual tokenizer. We present a self-supervised framework iBOT that
can perform masked prediction with an online tokenizer. Specifically, we
perform self-distillation on masked patch tokens and take the teacher network
as the online tokenizer, along with self-distillation on the class token to
acquire visual semantics. The online tokenizer is jointly learnable with the
MIM objective and dispenses with a multi-stage training pipeline where the
tokenizer needs to be pre-trained beforehand. We show the prominence of iBOT by
achieving an 82.3% linear probing accuracy and an 87.8% fine-tuning accuracy
evaluated on ImageNet-1K. Beyond the state-of-the-art image classification
results, we underline emerging local semantic patterns, which helps the models
to obtain strong robustness against common corruptions and achieve leading
results on dense downstream tasks, eg., object detection, instance
segmentation, and semantic segmentation.

18
The categories of this paper are: econ
11
The categories of this paper are: hep-lat
30
The categories of this paper are: q-alg
6
The categories of this paper are: astro-ph
4
The categories of this paper are: cond-mat
33
The categories of this paper are: plasm-ph
15
The categories of this paper are: nucl-ex
9
The categories of this paper are: nlin
17
The categories of this paper are: q-fin
7
The categories of this paper are: hep-th
32
The categories of this paper are: atom-ph
8
The categories of this paper are: hep-ex
29
The categories of this paper are: funct-an
27
The categories of this paper are: comp-gas
10
The categories of this paper are: q-bio
36
The categories of this paper are: cmp-lg