Our problem can be mapped onto a standard optimization problem in statistics. In particular, treating d as a latent variable, f as an observed variable, and q as a parameter vector, the optimization of P(f+|q) (or its loganthm) can be viewed as astandard maximum likelihood estimation problem for a latent variable model. It can be solved using the EM
agorithm (Dempster, Laird, & Rubin, 1977). The algorithm yields a sequence of variational parameters that monotonically increase the objective function log P(f+lq). Within the EM
framework, we obtain an update of the variational parameters by maximizing the expected complete log-likelihood:

$$\begin{array}{r c l}{{E}}&{{\log\,P(f^{+}|d,\,q)\,P(d)}}&{{=}}&{{E}}&{{\log\,P(f_{i}^{+}|d,\,q_{\,\|i})}}&{{+\,\mathrm{const\,at\,},}}\end{array}$$
$$(49)$$

w here qold denotes the vector of variational parameters before the update, where the constant term is independent of the variational parameters q and where the expectation is with respect to the posterior distribution P(d|f+,qold) œ P(f+|d,qold)P(d). Since the variational parameters associated with the conditional probabilities  P(f,*) d, q, i i ) are independent of one another, we can maximize each term in the above sum separately. Recalling the form of the variational transformation (see Eq. (24)), we have:

$$\begin{array}{r c l}{{E}}&{{\log\,P(f_{i}^{+\dagger}|d,\,q_{i}|_{\mathrm{{\scriptsize{I}}}})}}&{{=}}&{{q_{j|\mathrm{{\scriptsize{I}}}}\,E\{d_{j}\}}}&{{f}}&{{\theta_{i\,o}+\frac{\theta_{i\,j}}{q_{j}|_{\mathrm{{\scriptsize{I}}}}}}}&{{-\,f\left(\,\theta_{i\,o}\,\right)}}\\ {{}}&{{}}&{{}}&{{}}\\ {{}}&{{}}&{{}}&{{+f\left(\,\theta_{i\,o}\,\right)}}\end{array}$$

$$(50)$$

which we are to maximize with respect to q i j i while keeping the expectations E { d j } fixed. This optimization problem can be solved iteratively and monotonically by performing the following synchronous updates with normalization:

$$q_{j|\mathrm{{\scriptsize{\it~\leftarrow~}}}}\quad E\{d_{j}\}\quad q_{j|\mathrm{{\scriptsize{\it~\leftarrow~}}}}\ f\quad\theta_{i\,o}+{\frac{\theta_{i\,j}}{q_{j|\mathrm{{\scriptsize{\it~\leftarrow~}}}}}}\quad-\ \theta_{i\,j}\ f\quad\theta_{i\,o}+{\frac{\theta_{i\,j}}{q_{j|i}}}\quad-\ q_{j|\mathrm{{\scriptsize{\it~\leftarrow~}}}}\ f\,(\,\theta_{i\,o}\,)$$

$$\left(\,51\,\right)$$

where f' denotes the derivative of f. (The update is guaranteed to be non-negative).

This algorithm can be easily extended to handle the case where not all the positive findings have been transformed. The only new feature is that some of the conditional probabilities in the products P ( f + | d, g o id ) and P ( f + | d, q ) have been left intact, i.e., not transformed; the optimization with respect to the variational parameters corresponding to the transformed conditionals proceeds as before.

## Appendix C. Convexity

The purpose of this appendix is to demonstrate that the function:

$$\log\ E\quad e\quad{\mathrm{~}}{j,}{\mathrm{i}}\,\le\,m\ \xi\,{\mathrm{i}}\,\theta_{{\mathrm{i}}{\mathrm{j}}}\,d_{{\mathrm{j}}}$$
$$\left(52\right)$$

is a convex function of the variational parameters ξ i . We note first that affine transformations do not change convexity properties. Thus convexity in X =
jjj<m & 0jjdj implies