must be located at the end of the expansion. Thus in a PSDG, a policy might be expanded into a sequence of policies at the lower level which will be executed one after another before control is returned to the higher level policy. The implicit assumption here is that when a policy in the sequence terminates, it always does so at a state where the next policy in the sequence is applicable. Given this assumption, in the language of the AHMM we can define a compound policy π k as a policy that simply and orderly executes a sequence of policies at the lower level π ( 1 ) ; . . , π k − 1 ; independent of the current state. A PSDG is then equivalent to an AHMM if compound policies of this form are allowed.

Since the AMM closely follows the models used in abstract probabilistic planning, it can be used to model and recognise the behaviours of any autonomous agent whose decision making process is equivalent to an abstract MDP. It is also useful as a formal language for specifying contingent plans whose execution can then be monitored using the policy recognition algorithm. The language is also rich enough to specify a range of useful human behaviours, especially in domains where there is a natural hierarchical decomposition of the state space. Section 6 presents an application of the AHMM framework to the problem of recognising people behaviours in a complex spatial environment. Here, each policy of the AHMM represents the evolution of possible trajectories of people movement while the person performs a certain task in the environment such as heading towards a door, using the computer at a certain location, etc. The policies at different levels would represent the evolution of trajectories at different levels of abstraction. Due to the existing hierarchy in the domain, the policies can be constructed using the region-based decomposition of the state space. The environment is populated with multiple cameras divided into different zones that can provide the current location of the tracking target, albeit a noisy one. The noisy observations can be readily handled by the observation model in the AHMM. The policy recognition algorithm can then be applied to infer the person's current policy at different levels in the hierarchy.

One main restriction of the current AHMM model is that we consider only one toplevel policy at a time, thus are unable to model the inter-leaving of concurrent plans. Another more subtle restriction is the assumption that a high level policy selects the lower level policies depending only on the current state. If the state space is interpreted as the states of the external environment, this assumption implies that the actor either has full observation about the current state, or at least refines its intentions based on the actor's observation about the current state only (and not the entire observation history). Note that these restrictions of the AHMM also apply in the case of the PSDG model.

## 4. Dynamic Bayesian Network Representation

In this section, we describe the Dynamic Bayesian Network (DBN) representation of the AHMM. The network serves two purposes: (1) as the tool to derive the probabilistic independence property of this stochastic model, and (2) as the computational framework for the policy recognition algorithms in Section 5.

## 4.1 Network Construction

At time t, let s represent the current state, represent the current policy at level k
(k = 0,.., K), ef represent the ending status of π k , i.e., a boolean variable indicating