Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Field-aware Calibration: A Simple and Empirically Strong Method for
  Reliable Probabilistic Predictions   It is often observed that the probabilistic predictions given by a machine
learning model can disagree with averaged actual outcomes on specific subsets
of data, which is also known as the issue of miscalibration. It is responsible
for the unreliability of practical machine learning systems. For example, in
online advertising, an ad can receive a click-through rate prediction of 0.1
over some population of users where its actual click rate is 0.15. In such
cases, the probabilistic predictions have to be fixed before the system can be
deployed.
  In this paper, we first introduce a new evaluation metric named field-level
calibration error that measures the bias in predictions over the sensitive
input field that the decision-maker concerns. We show that existing post-hoc
calibration methods have limited improvements in the new field-level metric and
other non-calibration metrics such as the AUC score. To this end, we propose
Neural Calibration, a simple yet powerful post-hoc calibration method that
learns to calibrate by making full use of the field-aware information over the
validation set. We present extensive experiments on five large-scale datasets.
The results showed that Neural Calibration significantly improves against
uncalibrated predictions in common metrics such as the negative log-likelihood,
Brier score and AUC, as well as the proposed field-level calibration error.

33
The categories of this paper are: plasm-ph
4
The categories of this paper are: cond-mat
36
The categories of this paper are: cmp-lg
23
The categories of this paper are: solv-int
10
The categories of this paper are: q-bio
31
The categories of this paper are: ao-sci
26
The categories of this paper are: bayes-an
24
The categories of this paper are: dg-ga
17
The categories of this paper are: q-fin
25
The categories of this paper are: patt-sol
8
The categories of this paper are: hep-ex
15
The categories of this paper are: nucl-ex
34
The categories of this paper are: chem-ph
29
The categories of this paper are: funct-an
22
The categories of this paper are: chao-dyn
5
The categories of this paper are: gr-qc