DecisionListPolicy (Qa)
//Qa *is the set of Q-functions, one for each action;*
//Return the decision list policy ∆.

//Initialize decision list.

Let ∆ = {}.

//Compute the bonus functions.

For each action a, other than the default action d:
Compute the bonus for taking action a, δa(x) = Qa(x) − Qd(x);
as in Equation (19). Note that δa has scope restricted to Ta, as in Equation (20).

//Add states with positive bonuses to the (unsorted) decision list.

For each assignment t ∈ Ta:
If δa(t) > 0, add branch to decision list:
∆ = ∆ ∪ {ht**, a, δ**a(t)i}.

//Add the default action to the (unsorted) decision list.

Let ∆ = ∆ ∪ {h∅**, d,** 0i}.

//Sort decision list to obtain final policy.

Sort the decision list ∆ in decreasing order on the δ element of ht**, a, δ**i.

Return ∆.

Figure 8: Method for computing the decision list policy ∆ from the factored representation of the Qa **functions.**
In our example DBN, Ta2 = {X1, X2}.

Intuitively, we now have a situation where we have a "baseline" value function Qd(x)
which defines a value for each state x. Each action a **changes that baseline by adding or**
subtracting an amount from each state. The point is that this **amount depends only on** Ta, so that it is the same for all states in which the variables in Ta **take the same values.**
We can now define the greedy policy relative to our Q functions. For each action a**, define**
a set of conditionals ht, a, δi, where each t **is some assignment of values to the variables** Ta, and δ is δa(t**). Now, sort the conditionals for all of the actions by order of decreasing** δ:
ht1, a1, δ1i,ht2, a2, δ2i, . . . ,htL, aL, δLi.

Consider our optimal action in a state x**. We would like to get the largest possible "bonus"**
over the default value. If x is consistent with t1, we should clearly take action a1**, as it**
gives us bonus δ1. If not, then we should try to get δ2**; thus, we should check if** x is consistent with t2, and if so, take a2. Using this procedure, we can compute the decisionlist policy associated with our linear estimate of the value **function. The complete algorithm**
for computing the decision list policy is summarized in Figure 8.

Note that the number of conditionals in the list is Pa |Dom(Ta)|; Ta**, in turn, depends**
on the set of basis function clusters that intersect with the effects of a**. Thus, the size**
of the policy depends in a natural way on the interaction between the structure of our