

![0_image_0.png](0_image_0.png)

π ξ k are d-separated by π k and b in the new structure. Thus π λ k and π i ξ k are independent

## 5. Policy Recognition

In this section we begin to address the problem of policy recognition in the framework of the AHMM. We assume that a policy hierarchy is given and is modelled by an AHMM, however the top level policy and the details of its execution are unknown. The problem is then to determine the top level policy and other current policies at the lower levels given the current sequence of observations. In more concrete terms, we are interested in the conditional probability:
Pr(π k , . . , π k | õ t –1 )
and especially, the marginals:

## Pr(Π Ξ | Δ Ε –1), For All Levels K

Computing these probabilities gives us the information about the current policies at all levels of abstraction, from the current action ( k = 0), to the top-level policy ( k = K ),
taking into account all the observations that we have up to date.

In typical monitoring situations, these probabilities need to be computed "online", as each new observation becomes available. To do this, it is required to update the belief state (filtering distribution) of the AHMM at ea time point t. This problem is generally intractable unless the belief state has an efficient representation that affords a closed form update procedure. In our case, the belief state is a joint distribution of K + 3 discrete variables: Pr( π k t , . . , π f , s t , l | δ t ). Without any further structure imposed on the belief state, the complexity for updating it is exponential in K.

To cope with this complexity, one generally has to resort to some form of approximation to trade off accuracy for computational resources. On the other hand, the analysis of the