Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Ranking Models in Unlabeled New Environments   Consider a scenario where we are supplied with a number of ready-to-use
models trained on a certain source domain and hope to directly apply the most
appropriate ones to different target domains based on the models' relative
performance. Ideally we should annotate a validation set for model performance
assessment on each new target environment, but such annotations are often very
expensive. Under this circumstance, we introduce the problem of ranking models
in unlabeled new environments. For this problem, we propose to adopt a proxy
dataset that 1) is fully labeled and 2) well reflects the true model rankings
in a given target environment, and use the performance rankings on the proxy
sets as surrogates. We first select labeled datasets as the proxy.
Specifically, datasets that are more similar to the unlabeled target domain are
found to better preserve the relative performance rankings. Motivated by this,
we further propose to search the proxy set by sampling images from various
datasets that have similar distributions as the target. We analyze the problem
and its solutions on the person re-identification (re-ID) task, for which
sufficient datasets are publicly available, and show that a carefully
constructed proxy set effectively captures relative performance ranking in new
environments. Code is available at \url{https://github.com/sxzrt/Proxy-Set}.

26
The categories of this paper are: bayes-an
6
The categories of this paper are: astro-ph
4
The categories of this paper are: cond-mat
23
The categories of this paper are: solv-int
34
The categories of this paper are: chem-ph
33
The categories of this paper are: plasm-ph
25
The categories of this paper are: patt-sol
11
The categories of this paper are: hep-lat
29
The categories of this paper are: funct-an
28
The categories of this paper are: alg-geom
5
The categories of this paper are: gr-qc
37
The categories of this paper are: supr-con
7
The categories of this paper are: hep-th
15
The categories of this paper are: nucl-ex
31
The categories of this paper are: ao-sci
12
The categories of this paper are: nucl-th