it is found as the solution of:

  **for $\bf{x}$**:  $$\begin{array}{ll}\mbox{max}_{P({\bf x})}&-\sum_{\bf x}P({\bf x})\log P({\bf x}),\\ \mbox{s.t.}&\sum_{\bf x}P({\bf x})f_{i}({\bf x})=d_{i},\ (\forall i)\\ &\sum_{\bf x}P({\bf x})=1,\\ \mbox{\bf x}&\end{array}$$

where the last constraint ensures that P(x**) is properly normalized.**
Originally advocated in Jaynes (1957, 1982) as a generalization of Laplace's principle of indifference, the choice for the MaxEnt distribution can be defended in a variety of ways. The most common argument is that any distribution other than the MaxEnt distribution effectively makes additional assumptions about the data that reduce the entropy. As making additional assumptions biases the distribution in undue ways, the MaxEnt distribution is the safest bet.

A lesser known argument, but not less convincing, is a game-theoretic one
(Topsøe, 1979). Assuming that the true data distribution satisfies the given constraints, it remarks that the Shannon-optimal compression code **(e.g. Huffman)**
designed based on the MaxEnt distribution minimizes the worst-case **expected**
coding length of a message coming from the true distribution. Hence, using the MaxEnt distribution for designing a code is optimal in a robust minimax sense.

Besides these motivations for the MaxEnt principle, it is also relatively **easy**
to compute a MaxEnt model. Indeed, the MaxEnt optimization problem (2-4) is convex, and can be solved using standard techniques from convex **optimization**
theory (Boyd and Vandeberghe, 2004). Let us use Lagrange multiplier µ for constraint (4) and λi for constraints (3). Using λ **to denote the vector containing**
all Lagrange multipliers λi**, the Lagrangian is then equal to:**

$$L(\mu,\boldsymbol{\lambda},P(\mathbf{x}))=-\sum_{\mathbf{x}}P(\mathbf{x})\log P(\mathbf{x})$$ $$+\sum_{i}\lambda_{i}\left(\sum_{\mathbf{x}}P(\mathbf{x})f_{i}(\mathbf{x})-d_{i}\right)+\mu\left(\sum_{\mathbf{x}}P(\mathbf{x})-1\right).$$

$$\binom{5}{5}$$

Equating the derivative w.r.t. P(x**) to 0 yields the optimality conditions:**

log P(x) = µ − 1 +X
i
$$\begin{array}{r l}{={}}&{{}\mu-1+\sum_{i}\lambda_{i}f_{i}(\mathbf{x}),}\\ {\end{array}$$
$$\Leftrightarrow P({\bf x})\quad=\quad\frac{1}{Z}\exp\left(\sum_{i}^{i}\lambda_{i}f_{i}({\bf x})\right),$$
where we introduced a new variable Z = exp(1 − µ). The normalization constraint Px P(x) = 1 is often imposed constructively by choosing Z **to be an**
appropriate function of the other Lagrange multipliers λ**, in particular:**

$$\begin{array}{r c l}{{Z(\lambda)}}&{{=}}&{{\sum_{\mathbf{x}}\exp\left(\sum_{i}\lambda_{i}f_{i}(\mathbf{x})\right).}}\end{array}$$

$$\left({\mathfrak{h}}\right)$$

6