# Variational Probabilistic Inference And The Qmr-Dt Network

T OMMI @AI MIT .EDU
Tommi S. Jaakkola Artificial Intelligence Laboratory, Massachusetts In stitute of Technology, Cambridge, MA 02139 USA Michael I. Jordan Computer Science Division and Department of Statistics, University of California, Berkeley, CA 94720-1776 USA
J ORDAN @CS BERKELEY EDU

## Abstract

We describe a variational approximation method for efficient inference in large-scale probabilistic models. Variational methods are deterministic procedures that provide approximations to marginal and conditional probabilities of interest. They provide alternatives to approximate inference methods based on stochastic sampling or search. We describe a variational approach to the problem of diagnostic inference in the "Quick Medical Reference" (QMR) network. The QMR network is a large-scale probabilistic graphical model built on statistical and expert knowledge. Exact probabilistic inference is infeasible in this model for all but a small set of cases. We evaluate our variational inference algorithm on a large set of diagnostic test cases, comparing the algorithm to a state-of-the-art stochastic smmmmm

## 1. Introduction

Probabilistic models have become increasingly prevalent in Al in recent years. Beyond the significant representational advantages of probability theory, including guarantees of consistency and a naturalness at combining diverse sources of knowledge (Pearl, 1988), the discovery of general exact inference algorithms has been principally responsible for the rapid growth in probabilistic AI (see, e.g., Lauritzen & Spiegelhalter, 1988; Pearl, 1988; Shenoy, 1992). These exact inference methods greatly expand the range of models that can be treated within the probabilistic framework and provide a unifying perspective on the general problem of probabilistic computation in graphical models.

Probability theory can be viewed as a combinatorial calculus that instructs us in how to merge the probabilities of sets of events into probabilities of composites. The key operation is that of marginalization, which involves summing (or integrating) over the values of variables. Exact inference algonthms essentially find ways to perform as few sums as possible during marginalization operations. In terms of the graphical representation of probability distributions in which random variables correspond to nodes and conditional independencies are expressed as missing edges between nodes  exact inference algonthms define a notion of "locality" (for example as cliques in an appropriately defined graph), and attempt to restrict summation operators to locally defined sets of nodes.