  Inferring the value of a property of a large stochastic system is a difficult
task when the number of samples is insufficient to reliably estimate the
probability distribution. The Bayesian estimator of the property of interest
requires the knowledge of the prior distribution, and in many situations, it is
not clear which prior should be used. Several estimators have been developed so
far, in which the proposed prior was individually tailored for each property of
interest; such is the case, for example, for the entropy, the amount of mutual
information, or the correlation between pairs of variables. In this paper we
propose a general framework to select priors, valid for arbitrary properties.
We first demonstrate that only certain aspects of the prior distribution
actually affect the inference process. We then expand the sought prior as a
linear combination of a one-dimensional family of indexed priors, each of which
is obtained through a maximum entropy approach with constrained mean value of
the property under study. In many cases of interest, only one or very few
components of the expansion turn out to contribute to the Bayesian estimator,
so it is often valid to only keep a single component. The relevant component is
selected by the data, so no handcrafted priors are required. We test the
performance of this approximation with a few paradigmatic examples, and show
that it performs well in comparison to the ad-hoc methods previously proposed
in the literature. Our method highlights the connection between Bayesian
inference and equilibrium statistical mechanics, since the most relevant
component of the expansion can be argued to be the one with the right
temperature.
