Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: IMDB-WIKI-SbS: An Evaluation Dataset for Crowdsourced Pairwise
  Comparisons   Today, comprehensive evaluation of large-scale machine learning models is
possible thanks to the open datasets produced using crowdsourcing, such as
SQuAD, MS COCO, ImageNet, SuperGLUE, etc. These datasets capture objective
responses, assuming the single correct answer, which does not allow to capture
the subjective human perception. In turn, pairwise comparison tasks, in which
one has to choose between only two options, allow taking peoples' preferences
into account for very challenging artificial intelligence tasks, such as
information retrieval and recommender system evaluation. Unfortunately, the
available datasets are either small or proprietary, slowing down progress in
gathering better feedback from human users. In this paper, we present
IMDB-WIKI-SbS, a new large-scale dataset for evaluating pairwise comparisons.
It contains 9,150 images appearing in 250,249 pairs annotated on a
crowdsourcing platform. Our dataset has balanced distributions of age and
gender using the well-known IMDB-WIKI dataset as ground truth. We describe how
our dataset is built and then compare several baseline methods, indicating its
suitability for model evaluation.

17
The categories of this paper are: q-fin
37
The categories of this paper are: supr-con
31
The categories of this paper are: ao-sci
2
The categories of this paper are: math
3
The categories of this paper are: physics
8
The categories of this paper are: hep-ex
18
The categories of this paper are: econ
11
The categories of this paper are: hep-lat
30
The categories of this paper are: q-alg
27
The categories of this paper are: comp-gas
9
The categories of this paper are: nlin
6
The categories of this paper are: astro-ph
7
The categories of this paper are: hep-th
4
The categories of this paper are: cond-mat
32
The categories of this paper are: atom-ph
23
The categories of this paper are: solv-int