linguistic:translation
Exponential families

The log-likelihood is also particularly useful for exponential families of distributions, which include many of the common parametric probability distributions. The probability distribution function (and thus likelihood function) for exponential families contain products of factors involving exponentiation. The logarithm of such a function is a sum of products, again easier to differentiate than the original function.

An exponential family is one whose probability density function is of the form (for some functions, writing ⟨ −, − ⟩ {\textstyle \langle -,-\rangle } for the inner product):

p (x ∣ θ) = h (x) exp (⟨ η (θ), T (x) ⟩ − A (θ)). {\displaystyle p(x\mid {\boldsymbol {\theta }})=h(x)\exp {\Big (}\langle {\boldsymbol {\eta }}({\boldsymbol {\theta }}),\mathbf {T} (x)\rangle -A({\boldsymbol {\theta }}){\Big)}.}

Each of these terms has an interpretation, but simply switching from probability to likelihood and taking logarithms yields the sum:

ℓ (θ ∣ x) = ⟨ η (θ), T (x) ⟩ − A (θ) + log h (x). {\displaystyle \ell ({\boldsymbol {\theta }}\mid x)=\langle {\boldsymbol {\eta }}({\boldsymbol {\theta }}),\mathbf {T} (x)\rangle -A({\boldsymbol {\theta }})+\log h(x).}

The η (θ) {\textstyle {\boldsymbol {\eta }}({\boldsymbol {\theta }})} and h (x) {\textstyle h(x)} each correspond to a change of coordinates, so in these coordinates, the log-likelihood of an exponential family is given by the simple formula:

ℓ (η ∣ x) = ⟨ η, T (x) ⟩ − A (η). {\displaystyle \ell ({\boldsymbol {\eta }}\mid x)=\langle {\boldsymbol {\eta }},\mathbf {T} (x)\rangle -A({\boldsymbol {\eta }}).}

In words, the log-likelihood of an exponential family is inner product of the natural parameter η {\displaystyle {\boldsymbol {\eta }}} and the sufficient statistic T (x) {\displaystyle \mathbf {T} (x)}, minus the normalization factor (log-partition function) A (η) {\displaystyle A({\boldsymbol {\eta }})}. Thus for example the maximum likelihood estimate can be computed by taking derivatives of the sufficient statistic T and the log-partition function A.