

![0_image_0.png](0_image_0.png)

## 5.2 Policy Recognition: The General Case

We now return to the general case of policy recognition, i.e., without the two assumptions of the previous subsection. This makes the inference tasks in the AHMM much more difficult. Since neither the starting times nor the starting states of the current policies are known with certainty, theorem 1 cannot be used. Thus, the set of current policies no longer forms a chain structure as it did in C t since the conditional independence properties of the current time-slice no longer hold. We therefore cannot hope to represent the belief state by a simple structure as we did previously. An exact method for updating the belief state will thus have to operate on a structure with size exponential in K i and is bound to be intractable when K is large.

To cope with this complexity, an approximation scheme such as sequential importance sampling (SIS) (Doucet et al., 2000b; Liu & Chen, 1998; Kanazawa et al., 1995) can be employed. In our previous work (Bui, Venkatesh, & West, 1999), we have applied an SIS
method known as the likelihood weighting with evidence reversal (LW-ER) (Kanazawa et al., 1995) to an AHMM-like network structure. However the SIS method needs to sample in the product space of all the layers of the AHMM and thus becomes less accurate and inefficient with large K. The key to get around this inefficiency is to utilise the special structure of the AHMM, particularly, its special tractable case, to keep the set of variables that need to be sampled to a minimum.

The improvement of the SIS method to achieve this is has been presented in subsection 2.5 in the name of the Rao-Blackwellised SIS (RB-SIS) method. Rao-Blackwellisation specifically allows the marginalisation of some variables analytically and only samples the remaining variables. As a result, this reduces the averaged error, measured as the variance of the estimator (Casella & Robert, 1996).