is expected to have some net benefit (Dorais et al., 1998; Barber, Goel, & Martin, 2000a; Hexmoor & Kortenkamp, 2000).

A central problem in AA is to determine whether and when transfers of decision-making control should occur. A key challenge is to balance two potentially conflicting goals.  On the one hand, to ensure that the highest-quality decisions are made, an agent can transfer control to a human user (or another agent) whenever that user has superior decision-making expertise. 1 On the other hand, interrupting a user has high costs and the user may be unable to make and communicate a decision, thus such transfers-of-control should be minimized. Previous work has examined several different techniques that attempt to balance these two conflicting goals and thus address the transfer-of-control problem. For example, one technique suggests that decision-making control should be transferred if the expected utility of doing so is higher than the expected utility of making an autonomous decision
(Horvitz, Jacobs, & Hovel, 1999). A second technique uses uncertainty as the sole rationale for deciding who should have control, forcing the agent to relinquish control to the user whenever uncertainty is high (Gunderson & Martin, 1999). Yet other techniques transfer control to a user if an erroneous autonomous decision could cause significant harm (Dorais et al, 1998) or if the agent lacks the capability to make the decision (Ferguson, Allen, &
Miller, 1996).

Unfortunately, these previous approaches to transfer-of-control reasoning and indeed most previous work in AA, have focused on domains involving a single agent and a single user, isolated from interactions with other entities. When applied to interacting teams of agents and humans, where interaction between an agent and a human impacts the interaction with other entities, these techniques can lead to dramatic failures. In particular, the presence of other entities as team members introduces a third goal of maintaining coordination (in addition to the two goals already mentioned above), which these previous techniques fail to address. Failures occur for two reasons. Firstly, these previous techniques ignore team related factors, such as costs to the team due to incorrect decisions or due to delays in decisions during such transfers-of-control. Secondly (and more importantly), these techniques use one-shot transfers-of-control, rigidly committing to one of two choices: (i)
transfer control and wait for input (choice H ) or (ii) act autonomously (choice A ). However, given interacting teams of agents and humans, either choice can lead to costly failures if the entity with control fails to make or report a decision in a way that maintains coordination.

For instance, a human user might be unable to provide the required input due to a temporary communication failure; this may cause an agent to fail in its part of a joint action, as this joint action may be dependent on the user's input. On the other hand, forcing a less capable entity to make a decision simply to avoid miscoordination can lead to poor decisions with significant consequences. Indeed, as seen in Section 2.2, when we applied a rigid transfer-of-control decision-making to a domain involving teams of agents and users, it failed dramatically.

Yet, many emerging applications do involve multiple agents and multiple humans acting cooperatively towards joint goals. To address the shortcomings of previous AA work in such domains, this article introduces the notion of a transfer-of-control strategy. A transfer-ofcontrol strategy consists of a pre-defined, conditional sequence of two types of actions: (i)