information:classification
Description

Assume that we want to estimate an unobserved population parameter θ {\displaystyle \theta } on the basis of observations x {\displaystyle x}. Let f {\displaystyle f} be the sampling distribution of x {\displaystyle x}, so that f (x ∣ θ) {\displaystyle f(x\mid \theta)} is the probability of x {\displaystyle x} when the underlying population parameter is θ {\displaystyle \theta }. Then the function:

is known as the likelihood function and the estimate:

is the maximum likelihood estimate of θ {\displaystyle \theta }.

Now assume that a prior distribution g {\displaystyle g} over θ {\displaystyle \theta } exists. This allows us to treat θ {\displaystyle \theta } as a random variable as in Bayesian statistics. We can calculate the posterior distribution of θ {\displaystyle \theta } using Bayes' theorem:

where g {\displaystyle g} is density function of θ {\displaystyle \theta }, Θ {\displaystyle \Theta } is the domain of g {\displaystyle g}.

The method of maximum a posteriori estimation then estimates θ {\displaystyle \theta } as the mode of the posterior distribution of this random variable:

The denominator of the posterior distribution (so-called marginal likelihood) is always positive and does not depend on θ {\displaystyle \theta } and therefore plays no role in the optimization. Observe that the MAP estimate of θ {\displaystyle \theta } coincides with the ML estimate when the prior g {\displaystyle g} is uniform (i.e., g {\displaystyle g} is a constant function).

When the loss function is of the form

as c {\displaystyle c} goes to 0, the Bayes estimator approaches the MAP estimator, provided that the distribution of θ {\displaystyle \theta } is quasi-concave. But generally a MAP estimator is not a Bayes estimator unless θ {\displaystyle \theta } is discrete.