Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: ImageNet-21K Pretraining for the Masses   ImageNet-1K serves as the primary dataset for pretraining deep learning
models for computer vision tasks. ImageNet-21K dataset, which is bigger and
more diverse, is used less frequently for pretraining, mainly due to its
complexity, low accessibility, and underestimation of its added value. This
paper aims to close this gap, and make high-quality efficient pretraining on
ImageNet-21K available for everyone. Via a dedicated preprocessing stage,
utilization of WordNet hierarchical structure, and a novel training scheme
called semantic softmax, we show that various models significantly benefit from
ImageNet-21K pretraining on numerous datasets and tasks, including small
mobile-oriented models. We also show that we outperform previous ImageNet-21K
pretraining schemes for prominent new models like ViT and Mixer. Our proposed
pretraining pipeline is efficient, accessible, and leads to SoTA reproducible
results, from a publicly available dataset. The training code and pretrained
models are available at: https://github.com/Alibaba-MIIL/ImageNet21K

33
The categories of this paper are: plasm-ph
28
The categories of this paper are: alg-geom
2
The categories of this paper are: math
36
The categories of this paper are: cmp-lg
25
The categories of this paper are: patt-sol
32
The categories of this paper are: atom-ph
27
The categories of this paper are: comp-gas
37
The categories of this paper are: supr-con
7
The categories of this paper are: hep-th
15
The categories of this paper are: nucl-ex
5
The categories of this paper are: gr-qc
11
The categories of this paper are: hep-lat
20
The categories of this paper are: acc-phys
0
The categories of this paper are: hep-ph
17
The categories of this paper are: q-fin
35
The categories of this paper are: mtrl-th