Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Top-label calibration and multiclass-to-binary reductions   We investigate the relationship between commonly considered notions of
multiclass calibration and the calibration algorithms used to achieve these
notions, leading to two broad contributions. First, we propose a new and
arguably natural notion of top-label calibration, which requires the reported
probability of the most likely label to be calibrated. Along the way, we
highlight certain philosophical issues with the closely related and popular
notion of confidence calibration. Second, we outline general 'wrapper'
multiclass-to-binary (M2B) algorithms that can be used to achieve confidence,
top-label, and class-wise calibration, using underlying binary calibration
routines. Our wrappers can also be generalized to other notions of calibration,
if required for certain practical applications. We instantiate these wrappers
with the binary histogram binning (HB) algorithm, and show that the overall
procedure has distribution-free calibration guarantees. In an empirical
evaluation, we find that with the right M2B wrapper, HB performs significantly
better than other calibration approaches. Code for this work has been made
publicly available at https://github.com/aigen/df-posthoc-calibration.

30
The categories of this paper are: q-alg
18
The categories of this paper are: econ
0
The categories of this paper are: hep-ph
4
The categories of this paper are: cond-mat
11
The categories of this paper are: hep-lat
22
The categories of this paper are: chao-dyn
13
The categories of this paper are: quant-ph
14
The categories of this paper are: math-ph
10
The categories of this paper are: q-bio
7
The categories of this paper are: hep-th
9
The categories of this paper are: nlin
28
The categories of this paper are: alg-geom
20
The categories of this paper are: acc-phys
37
The categories of this paper are: supr-con
31
The categories of this paper are: ao-sci
34
The categories of this paper are: chem-ph