Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Masked Autoencoders Are Scalable Vision Learners   This paper shows that masked autoencoders (MAE) are scalable self-supervised
learners for computer vision. Our MAE approach is simple: we mask random
patches of the input image and reconstruct the missing pixels. It is based on
two core designs. First, we develop an asymmetric encoder-decoder architecture,
with an encoder that operates only on the visible subset of patches (without
mask tokens), along with a lightweight decoder that reconstructs the original
image from the latent representation and mask tokens. Second, we find that
masking a high proportion of the input image, e.g., 75%, yields a nontrivial
and meaningful self-supervisory task. Coupling these two designs enables us to
train large models efficiently and effectively: we accelerate training (by 3x
or more) and improve accuracy. Our scalable approach allows for learning
high-capacity models that generalize well: e.g., a vanilla ViT-Huge model
achieves the best accuracy (87.8%) among methods that use only ImageNet-1K
data. Transfer performance in downstream tasks outperforms supervised
pre-training and shows promising scaling behavior.

20
The categories of this paper are: acc-phys
33
The categories of this paper are: plasm-ph
31
The categories of this paper are: ao-sci
21
The categories of this paper are: adap-org
0
The categories of this paper are: hep-ph
13
The categories of this paper are: quant-ph
30
The categories of this paper are: q-alg
12
The categories of this paper are: nucl-th
37
The categories of this paper are: supr-con
9
The categories of this paper are: nlin
6
The categories of this paper are: astro-ph
22
The categories of this paper are: chao-dyn
26
The categories of this paper are: bayes-an
34
The categories of this paper are: chem-ph
8
The categories of this paper are: hep-ex
35
The categories of this paper are: mtrl-th