Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Testing whether a Learning Procedure is Calibrated   A learning procedure takes as input a dataset and performs inference for the
parameters $\theta$ of a model that is assumed to have given rise to the
dataset. Here we consider learning procedures whose output is a probability
distribution, representing uncertainty about $\theta$ after seeing the dataset.
Bayesian inference is a prime example of such a procedure, but one can also
construct other learning procedures that return distributional output. This
paper studies conditions for a learning procedure to be considered calibrated,
in the sense that the true data-generating parameters are plausible as samples
from its distributional output. A learning procedure whose inferences and
predictions are systematically over- or under-confident will fail to be
calibrated. On the other hand, a learning procedure that is calibrated need not
be statistically efficient. A hypothesis-testing framework is developed in
order to assess, using simulation, whether a learning procedure is calibrated.
Several vignettes are presented to illustrate different aspects of the
framework.

7
The categories of this paper are: hep-th
32
The categories of this paper are: atom-ph
27
The categories of this paper are: comp-gas
33
The categories of this paper are: plasm-ph
0
The categories of this paper are: hep-ph
11
The categories of this paper are: hep-lat
10
The categories of this paper are: q-bio
24
The categories of this paper are: dg-ga
22
The categories of this paper are: chao-dyn
28
The categories of this paper are: alg-geom
31
The categories of this paper are: ao-sci
5
The categories of this paper are: gr-qc
15
The categories of this paper are: nucl-ex
35
The categories of this paper are: mtrl-th
26
The categories of this paper are: bayes-an
20
The categories of this paper are: acc-phys