Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Surrogate Scoring Rules   Strictly proper scoring rules (SPSR) are incentive compatible for eliciting
information about random variables from strategic agents when the principal can
reward agents after the realization of the random variables. They also quantify
the quality of elicited information, with more accurate predictions receiving
higher scores in expectation. In this paper, we extend such scoring rules to
settings where a principal elicits private probabilistic beliefs but only has
access to agents' reports. We name our solution \emph{Surrogate Scoring Rules}
(SSR). SSR build on a bias correction step and an error rate estimation
procedure for a reference answer defined using agents' reports. We show that,
with a single bit of information about the prior distribution of the random
variables, SSR in a multi-task setting recover SPSR in expectation, as if
having access to the ground truth. Therefore, a salient feature of SSR is that
they quantify the quality of information despite the lack of ground truth, just
as SPSR do for the setting \emph{with} ground truth. As a by-product, SSR
induce \emph{dominant truthfulness} in reporting. Our method is verified both
theoretically and empirically using data collected from real human forecasters.

23
The categories of this paper are: solv-int
24
The categories of this paper are: dg-ga
33
The categories of this paper are: plasm-ph
9
The categories of this paper are: nlin
16
The categories of this paper are: stat
25
The categories of this paper are: patt-sol
7
The categories of this paper are: hep-th
6
The categories of this paper are: astro-ph
19
The categories of this paper are: eess
22
The categories of this paper are: chao-dyn
28
The categories of this paper are: alg-geom
15
The categories of this paper are: nucl-ex
20
The categories of this paper are: acc-phys
8
The categories of this paper are: hep-ex
4
The categories of this paper are: cond-mat
21
The categories of this paper are: adap-org