
| controlling-entity   | Action       |        |      |                 |
|----------------------|--------------|--------|------|-----------------|
| time                 | e; -response | Reward |      |                 |
| W(k + 1) - W(k)      |              |        |      |                 |
| e                    | t k          | no     | C    |                 |
| e                    | t k          | no     | wait | W(k + 1) - W(k) |
| t h                  | D            |        |      |                 |
| e                    | no           |        |      |                 |
| yes                  |              |        |      |                 |

transfer-of-control strategy.  Note that this reward function is described in an abstract fashion—for example, it does not specify how to compute the agent's expected quality of

## 4.2  Mdp  Representation Of Aa Problem Within Team Context

We have now given a high-level description of an MDP for implementing the notion of transfer-of-control strategies for AA. The remainder of this section provides a more detailed look at the MDP for a broad class of AA domains (including the E-Elves) where the agent acts on behalf of a user who is filling a role, ρ, within the context of a team activity, α . The reward function compares the EU of different strategies, finding the optimal one for the current state. To facilitate this calculation, we need to represent the parameters used in the model. We introduce the following state features to capture the aspects of the AA
problem in a team context:
- team-orig-expect-ρ is what the team originally expected of the fulfilling of ρ .

- team-expect-ρ is the team's current expectations of what fulfilling the role ρ implies.

- a gent-expect-p is the agent's (probabilistic) estimation for how ρ will be fulfilled.

- "other a attributes" encapsulate other aspects of the joint activity that are affected by the decision.
When we add these more specific features to the generic AA state features already presented, the overall state, within the MDP representation of a decision d, is a tuple:
(controlling-entity, team-orig-expect-p, team-expect-p, a gent-expect-p, α-status, e,-response, time, other α attributes)
For example, for a meeting scenario, team-orig-expect-o could be "Meet at 3pm", teamexpect- ρ could be "Meet at 3:15pm" after a user requested a delay, and a gent-expect- ρ could be "Meet at 3:30pm" if the agent believes its user will not make the rescheduled meeting.

The transition probability function for the AA MDP in a team context includes our underlying AA transition probabilities from Table 3, but it must also include probabilities oer these new state features. In particular, in addition to the temporal effect of the D action described in Section 4.1, there is the additional effect on the coordination of α .

The D action changes the value of the team-expect-ρ feature (in a domain-dependent but