A Matlab Program to Calculate the Maximum Entropy Distributions
Ali Mohammad-Djafari
The classical \ME \,(ME) problem consists of determining a probability distribution function (pdf) from a finite set of expectations $\mu_n = \efin$ of known functions $\finn$. The solution depends on $N+1$ Lagrange multipliers which are determined by solving the set of nonlinear equations formed by the $N$ data constraints and the normalization constraint. In this short communication we give three Matlab programs to calculate these Lagrange multipliers. The first considers the case where $\fin$ can be any functions. The second considers the special case where $\fin = x^n, \nn$. In this case the $\mu_n$ are the geometrical moments of $p(x)$. The third considers the special case where $\fin =$ exp$(-j n \omega x),\nn$. In this case the $\mu_n$ are the trigonometrical moments (Fourier components) of $p(x)$. We give also some examples to illustrate the usefullness of these programs.
formules.tex
%Formule
%LaTex

% My definitions
\def\AMD{Ali Mohammad-Djafari}
\def\LSS{Laboratoire des Signaux et Syst\`emes (CNRS-ESE-UPS) \par
\'Ecole Sup\'erieure d'\'Electricit\'e \par
Plateau de Moulon, 91192 Gif sur Yvette C\'edex, France. \par}

\def\ME{maximum entropy}
\def\pdf{probability distribution function}
\def\lm{Lagrange multipliers}
%
\def\beq{\begin{equation}}
\def\eeq{\end{equation}}

\def\beqn{\begin{eqnarray}}
\def\eeqn{\end{eqnarray}}

\def\beqnn{\begin{eqnarray*}}
\def\eeqnn{\end{eqnarray*}}

\def\barr{\begin{array}}
\def\earr{\end{array}}

\def\bit{\begin{itemize}}
\def\eit{\end{itemize}}

\def\ben{\begin{enumerate}}
\def\een{\end{enumerate}}

\def\bpic{\begin{picture}}
\def\epic{\end{picture}}

\def\bfig{\begin{figure}}
\def\efig{\end{figure}}

\def\bver{\begin{verbatim}}
\def\ever{\end{verbatim}}

\def\bc{\begin{center}}
\def\ec{\end{center}}

\def\br{\begin{right}}
\def\er{\end{right}}

\def\bl{\begin{left}}
\def\el{\end{left}}
%
\def\fix#1{\phi _#1(x)}
\def\xvec#1#2#3{\{#1_#2,\ldots,#1_#3\}}

\def\bm#1{\mbox{\boldmath $#1$}}
\def\xb {{\bm x}}
\def\yb {{\bm y}}
\def\bb {{\bm b}}
\def\Ab {{\bm A}}
\def\vb {{\bm v}}
\def\Gb {{\bm G}}
\def\xbh{\hat{{\bm x}}}
\def\pyx{p(\yb \mid \xb)}
\def\pxy{p(\xb \mid \yb)}

\def\dx{\,{\rm d}x}

\def\fin{\fix n}
\def\fik{\fix k}
\def\fiz{\fix 0}
\def\sfinz{\sum_{n=0}^N \lambda_n \, \fin}
\def\sfinu{\sum_{n=1}^N \lambda_n \, \fin}
\def\bl{\bm{\lambda}}
\def\bd{\bm{\delta}}
\def\blz{\bl ^0}
\def\gnl{G _n(\bl)}
\def\gnlz{G _n(\blz)}
\def\un{0,\dots, N}
\def\nn{n=\un}
\def\finn{\fin , \nn}
\def\esfinz{\exp\,\left[ -\sfinz \right] }
\def\esfinu{\exp\,\left[ -\sfinu \right] }
\def\esxm{\exp\,\left[ -\sum_{m=0}^N \lambda_m \, x^m \right] }
\def\esp#1{{\rm E}\left\lbrace #1 \right\rbrace}
\def\efin{\esp \fin }
\def\vlambda{\bm{\lambda} = [\lambda_0,\ldots,\lambda_n]}
\def\zl{Z(\bl)}
\def\finxi{\phi _n(x_i)}
\def\snfinxi{\sum_{n=1}^N \lambda_n \finxi}
\def\esnfinxi{\exp \left[ - \snfinxi \right]}
\def\smfinxi{\sum_{i=1}^M \finxi}

\def\ejnw{\exp \left( -j n \omega_0 x \right) }
\def\eejnw{{\rm E} \left\lbrace \ejnw \right\rbrace}
\def\table#1#2#3#4#5#6#7#8#9{#1 \quad #2 \quad #3 \quad #4 \quad #5 \quad #6 \quad #7 \quad #8 \quad #9}

\def\ent{-\int p(x) \, \ln p(x) \dx}
\def\dfdx#1#2{\frac{\partial #1}{\partial #2}}

\def\signed#1{{\unskip\nobreak\hfil\penalty50\hskip2em\hbox{}
\nobreak\hfil\tt#1\parfillskip=0pt \finalhyphendemerits=0 \par}}

\def\uncatcodespecials{\def\do##1{\catcode`##1=12 }\dospecials}
\def\listing#1{\par\begingroup\setupverbatim\input{#1} \endgroup}
\newcount\lineno
\def\setupverbatim{\tt\tiny \lineno=0
 \obeylines \uncatcodespecials \obeyspaces
 \everypar{\advance\lineno by1 \llap{\the\lineno\ \ }}}
{\obeyspaces\global\let =\ }

\def\pmatrix#1{\left[\begin{matrix} #1\end{matrix}\right]}
\def\eqalign#1{\left\{\begin{array}{lcl} #1\end{array}\right.}
\def\cases#1{\left\{\begin{array}{lcl} #1\end{array}\right.}

maxent91_1.tex
% LaTex file
% Date 2 Septembre 1991
% A. Mohammad-djafari
% MaxEnt91_1.tex
% Needs FORMULES.TEX
%       ME_DENS1.M
%       ME_DENS2.M
%       ME_DENS3.M
%       ME1.M
%       ME2.M
%       ME3.M
%       FIN1_X.M
%       FIN3_X.M
%
\documentclass[nomathfonts]{aipproc} 
\layoutstyle{6s}
\usepackage{xspace,graphicx,amsmath,amssymb,bbm}
\graphicspath{{Eps/}}
%----------------------------------------------------------
\title{A Matlab Program to Calculate the Maximum Entropy Distributions}
\author{Ali Mohammad-Djafari}
{
  address = {Laboratoire des Signaux et Syst\`emes,\linebreak 
  Sup\'elec, Plateau de Moulon, 91192 Gif-sur-Yvette, France}
  ,email = djafari@lss.supelec.fr
}

%\documentclass[twoside,11pt]{maxent}
%\documentstyle[twoside,11pt]{maxent}

\title{A Matlab Program to Calculate the Maximum Entropy Distributions}

\begin{document}
\maketitle
\input formules.tex

\begin{abstract} 
The classical \ME \,(ME) problem consists of determining a
probability distribution function (pdf) from a finite set of 
expectations $\mu_n = \efin$  of known functions $\finn$. 
The solution depends on  $N+1$ Lagrange multipliers which are 
determined by solving the set of nonlinear equations formed by 
the $N$ data constraints and the normalization constraint. 
In this short communication we give three Matlab programs to
calculate these Lagrange multipliers. The first considers the case  
where $\fin$ can be any functions. 
The second considers the special case where $\fin = x^n, \nn$. 
In this case the $\mu_n$ are the geometrical moments of $p(x)$. 
The third considers the special case where $\fin =$ exp$(-j n \omega x),\nn$. 
In this case the $\mu_n$ are the trigonometrical moments 
(Fourier components) of $p(x)$. 
We give also some examples to illustrate the usefullness of these 
programs.
\end{abstract} 

\section{Introduction}
Shannon (1948) indicated how maximum entropy (ME) distributions 
can be derived by a straigtforward application of the calculus of 
variations technique. He defined the entropy of a probability density 
function $p(x)$ as
\begin{equation}
H = \ent
\end{equation}
Maximizing $H$ subject to various side conditions is well--known 
in the literature as a method for deriving the forms of minimal 
information prior distributions; e.g. Jaynes (1968) and 
Zellner (1977). Jaynes (1982) has extensively analyzed examples in 
the discrete case, while in Lisman and Van Znylen (1972), Rao (1973) and 
Gokhale (1975), Kagan, Linjik  continuous cases are considered. 
In the last case, the problem, in its general form, is the following
\begin{eqnarray}
 {\rm maximize}\quad H &=& \ent \nonumber \\
 \hbox {subject to} \quad \efin &=& \int{\fin \, p(x) \dx} = \mu_n , \quad \nn
\end{eqnarray}
where  $\mu_0=1$ , $\fiz =1$ and $\finn$ are $N$ known functions, 
and $\mu_n, \nn$ are the given expectation data. 
The classical solution of this problem is given by
\begin{equation}
p(x) = \esfinz
\end{equation}
The $(N + 1)$ Lagrangien parameters $\vlambda$  are obtained by 
solving the following set of $(N + 1)$ nonlinear equations
\begin{equation}
 \gnl = {\int \fin \, \esfinz \dx} = \mu_n, \quad \nn
 \label{eq.4}
\end{equation}
The distributions defined by (3) form a great number of known 
distributions which are obtained by choosing the appropriate $N$ 
and $\finn$. In general $\fin$ are either the powers of $x$ or 
the logarithm of $x$. See Mukhrejee and Hurst (1984), 
Zellner (1988), Mohammad--Djafari (1990) for many other examples 
and discussions. 
Special cases have been extensively analyzed and
used by many authors. When $\fin = x^n, \nn $ then $\mu_n , \nn $ 
are the given $N$ moments of the distribution. 
See, for example, Zellner (1988) for a numerical implementation 
in the case $N=4$. 

In this communication we propose three programs written in MATLAB 
to solve the system of equations~(\ref{eq.4}). 
The first is a general program where $\fin $ can be any
functions.  The second is a special case where  $\fin = x^n, \nn$. 
In this case the $\mu_n$  are the geometrical moments of $p(x)$. 
The third is a special case where  $\fin =$ exp$(-j n \omega x), \nn$. 
In this case the $\mu_n$ are the trigonometrical moments (Fourier 
components) of $p(x)$. We give also some examples to illustrate 
the usefullness of these programs.

\section{Principle of the method}

We have seen that the solution of the standard ME problem is given 
by (3) in which the Lagrange multipliers  $\bl$ are 
obtained by solving the nonlinear equations (4). 
In general, these equations are solved by the standard Newton 
method which consists of expanding $G_n(\bl)$ in Taylor's 
series around trial values of the {\sl lambda}'s, drop the quadratic 
and higher order terms, and solve the resulting linear system 
iteratively. We give here the details of the numerical
method  that we implemented. 
When developing the $G_n(\bl)$ in equations~(\ref{eq.4}) in first 
order Taylor's series around the trial  $\bl ^0$, the 
resulting linear equations are given by
\begin{equation}
 \gnl\cong\gnlz + (\bl - \blz)^t \left[\, {\rm {\bf grad}} \,
 G_n(\bl)\right]_{(\bl=\blz)}=\mu_n, \quad \nn
 \label{eq.5}
\end{equation}
Noting the vectors $\bm{\delta}$  and $\vb$ by
\[
\bm{\delta}  = \bl - \blz
\]
\[
{\bm v} = {\left[\mu_0 - G_0(\blz),\ldots,\mu_N - G_N(\blz)\right]}^t
\]
and the matrix $\Gb$ by
\begin{equation}
 \Gb = \left[\barr{c} ~\\ g_{nk}\\ ~\\ \earr\right]=
 \left[\barr{c} ~\\ \frac{\partial \gnl}{\partial \lambda_k}\\ ~\\ \earr\right]_{(\bl = \blz)} \quad n,k=\un
\end{equation}
then equations~(\ref{eq.5}) become 
\begin{equation}
 \Gb \, \bm{\delta}  \, = \, \vb
 \label{eq.7}
\end{equation}
This system is solved for $\bm{\delta}$  from which we drive 
$\bl = \blz + \bm{\delta}$,  
which becomes our new initial vector $\blz$ and the iterations  
continue until $\bm{\delta}$ becomes appropriately small. 
Note that the matrix $\Gb$ is a symmetric one and we have
\begin{equation}
 g_{nk}=g_{kn}=- {\int \fin \, \fik \, \esfinz \dx} 
 \quad n, k= \un
 \label{eq.8}
\end{equation}
So in each iteration we have to calculate the  $N(N - 1) / 2$
integrals in the  equation (8). The algorithm of the general 
Maximum Entropy problem is then  as follows:
\begin{enumerate}
	\item Define the range and the discretization step of $x$ 
       \signed {\tt (xmin, xmax,dx).}
 \item Write a function to calculate $\finn $ 
       \signed {\tt(fin\_x).}
	\item Start the iterative procedure with an initial estimate $\blz$ 
       \signed {\tt (lambda0).}
	\item Calculate the $(N+1)$ integrals in equations~(\ref{eq.4}) 
       and the $N (N-1)/2$ distinct elements $g_{nk}$ of the 
       matrix $\Gb$ by calculating the integrals in the equations(\ref{eq.8}) 
       \signed {\tt (Gn, gnk).}
	\item Solve the equation~(\ref{eq.7}) to find $\bm{\delta}$ 
       \signed {\tt (delta).}
	\item Calculate $\bl = \blz + \bm{\delta}$ and go back to step 3 until 
       $\bm{\delta}$ becomes negligible.
\end{enumerate}

{\noindent The calculus of the integrals in equations (4) and (8) 
can be made by a univariate Simpson's method. 
We have used a very simplified version of  this method.}

\subsection {Case of geometrical moments}
Now consider the special case of moments problem where 
$\fin = x^n,\quad \nn$.
In this case equations (3), (4) and (8) become
\begin{equation}
 p(x) = \esxm
\end{equation}
\begin{equation}
 \gnl = {\int x^n  \esxm \dx} =  \mu_n ,\quad \nn
 \label{eq.10}
\end{equation}
\begin{equation}
 g_{nk} = g_{kn} = -  {\int x^n x^k \esxm \dx} =  - G_{n+k} 
 (\bl)\quad n , k= \un
\end{equation}

This means that the $[(N+1) \times (N+1)]$ matrix $\Gb$ in equation~(\ref{eq.7}) 
becomes a symmetric Hankel matrix which is entirely defined 
by $2 N + 1$ values $\gnl, n = 0,\ldots, 2 N$. 
So the algorithm in this case is the same as in the precedent one 
with two simplifications

\begin{enumerate}
	\item In step 2 we do not need to write a seperate function 
       to calculate  the functions $\fin�= x^n, \nn$.
	\item In step 4 the number of integral evaluations is 
       reduced, because the elements $g_{nk}$  of the matrix $\Gb$ are 
       related to the integrals $\gnl$ in  equations~(\ref{eq.10}). 
       This matrix is defined entirely by only $2 N +1$  components.
\end{enumerate}

\subsection { Case of trigonometrical moments}
Another interesting special case is the case where 
the data are the Fourier  components of $p(x)$

\begin{equation}
 \eejnw = {\int \ejnw \, p(x) \dx} = \mu_n , \quad \nn ,
\end{equation}
where $\mu_n$ may be complex--valued and has the property  
$\mu_{-n} = \mu_n^{\*}$. 
This means that we have the following relations
\beqn
 \fin &=& \ejnw , \quad n=-N,\ldots ,0, \ldots N, \\
 p(x) &=& \exp \left[ 
 -  {\rm Real} \sum_{n=0}^N \lambda_n \, \ejnw 
\right] , \\
\gnl &=& \int \ejnw \, p(x) \dx, \quad \nn, \\
 g_{nk} &=& 
  \left\{\begin{array}{ll}
	 - G_{n-k} (\bl) & \mbox{for~~} n \geq k,\\
    - G_{n+k}^{\*} (\bl) & \mbox{for~~} n< k
  \end{array}\right.
  \quad n,k = \un,
\eeqn
so that all the elements of the matrix $\Gb$ are related to 
the discrete Fourier transforms of $p(x)$. Note that $\Gb$ is a Hermitian 
Toeplitz matrix.

\section{Examples and Numerical Experiments}
To illustrate the usefullness of the proposed programs we consider 
first the case of the Gamma distribution	
\begin{equation}
p(x;\alpha,\beta)=
\frac{\beta^{(1-\alpha)}}{\Gamma (1-\alpha)} \, 
x^\alpha \exp (-\beta x),
\quad x > 0, \alpha < 1, \beta  > 0 .
\end{equation}
This distribution can be considered as a ME distribution when the 
constraints are
\begin{eqnarray}
\left\{\begin{array}{ll}
 \int p(x;\alpha,\beta)\dx          &= 1     \\
 \int x \, p(x;\alpha,\beta)\dx     &= \mu_1 \\
 \int \ln (x) \, p(x;\alpha,\beta) \dx &= \mu_2 \\
\end{array}\right.
\quad
\left\{\begin{array}{ll}
 \phantom\int{\rm normalization}\quad \fix 0 &= 1, \\
 \phantom\int                    \fix 1 &= x,      \\
 \phantom\int                    \fix 2 &= \ln (x).\\
\end{array}\right.
\end{eqnarray}
This is easy to verify because the equation (12) can be written as
\[
 p(x;\alpha,\beta)=
 \exp \bigl[-\lambda_0 - \lambda_1 x -�\lambda_2 \ln (x) \bigr]
\]
\[
 {\rm with} \quad \lambda_0 = - \ln 
 {\frac{\beta^{(1-\alpha)}}{\Gamma (1-\alpha)}}
 , \quad \lambda_1 = \beta \quad {\rm and} \quad \lambda_2 = -\alpha .
\]
Now consider the following problem  

\begin{center}
 Given $\mu_1$ and $\mu_2$ determine $\lambda_0$, $\lambda_1$ and $\lambda_2$. 
\end{center}

\noindent This can be done by the standard ME method. 
To do this, first we must define the range of $x$, 
{\tt (xmin, xmax, dx)}, and write a function {\tt fin\_x}
to calculate the functions $\fix 0 = 1$, $\fix 1 = x$ and 
$\fix 2 = \ln x$  (See the function {\tt fin1\_x} in Annex).
Then we must define an initial estimate $\blz$ for $\bl$ and, 
finally, let the program works.

The case of the {\it Gamma} distribution is interesting because 
there is an analytic relation between $(\alpha, \beta)$ and the 
mean $m = \esp x$ and variance $\sigma ^2 = \esp {(x - m)^2}$ 
which is

\begin{equation}
 \left\{\begin{array}{ll}
  m=(1-\alpha) / \beta \\ \sigma^2 =(1-\alpha) / \beta^2 
 \end{array}\right.,
\end{equation}
or inversely
\begin{equation}
 \left\{\begin{array}{ll}
  \alpha = (\sigma^2 -m^2) / \sigma^2 \\ \beta = m / \sigma^2 ,
 \end{array}\right.,
\end{equation}
so that we can use these relations to determine $m$ and $\sigma^2$. 
Note also that the corresponding entropy of the final 
result is a byproduct of the function. 
Table (1) gives some numerical results obtained by  ME\_DENS1 
program (See Annex).

\smallskip
\begin{center}
\begin{tabular}{cccccc} \hline
& & & Table 1. & &\\ \hline 
  $\quad\mu_1$ & $\quad\mu_2$ & 
 $\quad\alpha$ &	$\quad\beta$ & 
 $\quad m$ & $\quad\sigma^2$\\\hline 
 0.2000 & -3.0000 & 0.2156 & -3.0962 & 0.2533 & 0.0818 \\
 0.2000 & -2.0000 &-0.4124 & -6.9968 & 0.2019 & 0.0289 \\
 0.3000 & -1.5000 &-0.6969 & -5.3493 & 0.3172 & 0.0593 \\
\hline
\end{tabular}
\end{center}

The next example is the case of a quartic distribution
\begin{equation}
 p(x) = \exp \left[-\sum_{n=0}^4 \lambda_n \, x^n \right].
\end{equation}
This distribution can be considered as a ME distribution when 
the constraints are
\begin{equation}
 \esp {x^n} = \int x^n \, p(x) \dx = \mu_n, \quad n = 0,\ldots,4 
 \quad {\rm with} \quad \mu_0 = 1.
\end{equation}
Now consider the following problem : 
Given $\mu_n, n = 1,\ldots, 4$ calculate  $\lambda_n, n = 0,\ldots, 4$ . 
This can be done by the ME\_DENS2 program. 
Table (2) gives some numerical results obtained by this program: 

\smallskip
\begin{center}
\begin{tabular}{cccccccccc} \hline
& & & & & Table 2. & &\\ \hline 
$\mu_1$ & $\mu_2$ & $\mu_3$ & $\mu_4$ & 
$\lambda_0$ & $\lambda_1$ & $\lambda_2$ & $\lambda_3$ & 
$\lambda_4$\\ \hline
 0 & 0.2 & 0.05 & 0.10 & 
0.1992 & 1.7599 & 2.2229 & -3.9375 & 0.4201\\ 
 0 & 0.3 & 0.00 & 0.15 & 
0.9392 & 0.000 & -3.3414 & 0.0000 & 4.6875\\ 
 0 & 0.3 & 0.00 & 0.15 & 
0.9392 & 0.000 & -3.3414 & 0.0000 & 4.6875\\ 
\hline
\end{tabular}
\end{center}

These examples show how to use the proposed programs. A third example is 
also given in Annex which shows how to use the ME\_DENS3 program 
which considers the case of trigonometric moments.

\section{Conclusions}
In this paper we addressed first the class of ME distributions 
when the available data are a finite set of expectations 
$\mu_n = \esp \fin$ of some known functions $\fin,\, \nn$. 
We proposed then three Matlab programs to solve this problem by a 
Newton--Raphson method in general case, in case of geometrical 
moments data where $\fin = x^n$ and in case of trigonometrical 
moments where $\fin = \ejnw$.
Finally, we gave some numerical results for some special examples who 
show how to use the proposed programs.

\begin{thebibliography}{99}

\bibitem{ZEL88} 
A. Zellnerr and R. Highfiled, 
``Calculation of Maximum Entropy Distributions and Approximation of 
Marginal Posterior Distributions'', 
{\it Journal of Econometrics\,} 
{\bf 37}, 1988, 195--209, North Holland.

\bibitem{MUK84} 
D. Mukherjee and D.C. Hurst, 
``Maximum Entropy Revisited'', 
{\it Statistica Neerlandica\,} {\bf 38}, 1984, n� 1, 1--12.

\bibitem{LAZ78} 
Verdugo Lazo and P.N. Rathie, 
``On the Entropy of Continuous Probability Distributions'', 
{\it IEEE Trans.\,}, {\bf vol. IT--24}, n� 1, 1978.

\bibitem{GOK75} 
Gokhale, 
``Maximum Entropy Characterizations of some distributions'', 
{\it Statistical distributions in Scientific work\,}, 
{\bf vol. 3}, 299--304 
(G.P. Patil et al., Eds., Reidel, Dordrecht, Holland, 1975).

\bibitem{JAY83} 
Jaynes, 
``Papers on probability, statistics and statistical physics'', 
{\it Reidel Publishing Company, Dordrecht\,}, 
Holland, 1983.

\bibitem{MAT78} 
Matz, 
``Maximum Likelihood parameter estimation for the quartic 
exponential distributions'', 
{\it Technometrics\,}, {\bf 20}, 475--484, 1978.

\bibitem{AMD90a}
Mohammad-Djafari A. et Demoment G., 
"Estimating Priors in Maximum Entropy Image Processing," 
{\it Proc. of ICASSP 1990\,}, pp: 2069-2072

\bibitem{AMD90b}
Mohammad-Djafari A. et Idier J., 
"Maximum entropy prior laws of images and estimation of their parameters," 
{\it Proc. of The 10th Int. MaxEnt Workshop, Laramie, Wyoming\,}, 
published in Maximum-entropy and Bayesian methods, T.W. Grandy ed., 1990.
\end{thebibliography}

\newpage
\section{Annex A}
\listing{me_dens1.m}

\newpage
\listing{me1.m}

\bigskip
\listing{fin1_x.m}

\newpage
\listing{me_dens2.m}

\bigskip
\listing{me2.m}

\newpage
\listing{me_dens3.m}

\newpage
\listing{me3.m}

\bigskip
\listing{fin3_x.m}

\end{document}

physics0111126.extracted_bib
\begin{thebibliography}{99}

\bibitem{ZEL88} 
A. Zellnerr and R. Highfiled, 
``Calculation of Maximum Entropy Distributions and Approximation of 
Marginal Posterior Distributions'', 
{\it Journal of Econometrics\,} 
{\bf 37}, 1988, 195--209, North Holland.

\bibitem{MUK84} 
D. Mukherjee and D.C. Hurst, 
``Maximum Entropy Revisited'', 
{\it Statistica Neerlandica\,} {\bf 38}, 1984, n� 1, 1--12.

\bibitem{LAZ78} 
Verdugo Lazo and P.N. Rathie, 
``On the Entropy of Continuous Probability Distributions'', 
{\it IEEE Trans.\,}, {\bf vol. IT--24}, n� 1, 1978.

\bibitem{GOK75} 
Gokhale, 
``Maximum Entropy Characterizations of some distributions'', 
{\it Statistical distributions in Scientific work\,}, 
{\bf vol. 3}, 299--304 
(G.P. Patil et al., Eds., Reidel, Dordrecht, Holland, 1975).

\bibitem{JAY83} 
Jaynes, 
``Papers on probability, statistics and statistical physics'', 
{\it Reidel Publishing Company, Dordrecht\,}, 
Holland, 1983.

\bibitem{MAT78} 
Matz, 
``Maximum Likelihood parameter estimation for the quartic 
exponential distributions'', 
{\it Technometrics\,}, {\bf 20}, 475--484, 1978.

\bibitem{AMD90a}
Mohammad-Djafari A. et Demoment G., 
"Estimating Priors in Maximum Entropy Image Processing," 
{\it Proc. of ICASSP 1990\,}, pp: 2069-2072

\bibitem{AMD90b}
Mohammad-Djafari A. et Idier J., 
"Maximum entropy prior laws of images and estimation of their parameters," 
{\it Proc. of The 10th Int. MaxEnt Workshop, Laramie, Wyoming\,}, 
published in Maximum-entropy and Bayesian methods, T.W. Grandy ed., 1990.
\end{thebibliography}