quality:correction
Restricted parameter space

While the domain of the likelihood function—the parameter space —is generally a finite-dimensional subset of Euclidean space, additional restrictions sometimes need to be incorporated into the estimation process. The parameter space can be expressed as

where h (θ) = {\displaystyle \;h(\theta)=\left\;} is a vector-valued function mapping R k {\displaystyle \,\mathbb {R} ^{k}\,} into R r. {\displaystyle \;\mathbb {R} ^{r}~.} Estimating the true parameter θ {\displaystyle \theta } belonging to Θ {\displaystyle \Theta } then, as a practical matter, means to find the maximum of the likelihood function subject to the constraint h (θ) = 0. {\displaystyle ~h(\theta)=0~.}

Theoretically, the most natural approach to this constrained optimization problem is the method of substitution, that is "filling out" the restrictions h 1, h 2, …, h r {\displaystyle \;h_{1},h_{2},\ldots,h_{r}\;} to a set h 1, h 2, …, h r, h r + 1, …, h k {\displaystyle \;h_{1},h_{2},\ldots,h_{r},h_{r+1},\ldots,h_{k}\;} in such a way that h ∗ = {\displaystyle \;h^{\ast }=\left\;} is a one-to-one function from R k {\displaystyle \mathbb {R} ^{k}} to itself, and reparameterize the likelihood function by setting ϕ i = h i (θ 1, θ 2, …, θ k). {\displaystyle \;\phi _{i}=h_{i}(\theta _{1},\theta _{2},\ldots,\theta _{k})~.} Because of the equivariance of the maximum likelihood estimator, the properties of the MLE apply to the restricted estimates also. For instance, in a multivariate normal distribution the covariance matrix Σ {\displaystyle \,\Sigma \,} must be positive-definite; this restriction can be imposed by replacing Σ = Γ T Γ, {\displaystyle \;\Sigma =\Gamma ^{\mathsf {T}}\Gamma \;,} where Γ {\displaystyle \Gamma } is a real upper triangular matrix and Γ T {\displaystyle \Gamma ^{\mathsf {T}}} is its transpose.

In practice, restrictions are usually imposed using the method of Lagrange which, given the constraints as defined above, leads to the restricted likelihood equations

where λ = T {\displaystyle ~\lambda =\left^{\mathsf {T}}~} is a column-vector of Lagrange multipliers and ∂ h (θ) T ∂ θ {\displaystyle \;{\frac {\partial h(\theta)^{\mathsf {T}}}{\partial \theta }}\;} is the k × r Jacobian matrix of partial derivatives. Naturally, if the constraints are not binding at the maximum, the Lagrange multipliers should be zero. This in turn allows for a statistical test of the "validity" of the constraint, known as the Lagrange multiplier test.