

![0_image_0.png](0_image_0.png)

team_auto2 to members of a subteam TEAM-ESCORT-FOLLOW (a subteam of ES-
CORTS). The ontent of this message indi ates that the subteam should establish ommitment to a plan named prepare-to-exe ute-mission. The online appendix presents sample

```
logs of the overheard messages from 
                                       omplete runs, as well as the plan and team hierar
                                                                                           hies for the eva
            uation appli
                           ation.
                                 

```

As dis ussed in Se tion 1, the apability for automati ally monitoring the progress of the team is riti al. This need for team monitoring is further amplied in distributed settings, sin e a human operator in one pla e annot dire tly observe the agents exe uting in a remote lo ation. For instan e, in trial runs of the eva uation simulation appli ation des ribed above, monitoring sometimes required a series of franti phone alls among human

```
operators in dierent states, trying to verify the su
                                                 
                                                  essful exe
                                                            ution of the system as it was
                                                                                       

```

operating. And even when this agent team was o-lo ated on multiple omputers in one

```
room, the diversity of agents made it extremely di
                                               ult for an observer to automati
                                                                           ally
                                                                              

```

monitor the state of the team just from observing the dierent agent output s reens.

Overseer was built to provide su h monitoring by tra king the routine ommuni ations among the agents (Figure 2). Using plan-re ognition, it allows humans and agents to query about the present and future likely plans of the entire team, its subteams and

```
individualsto monitor progress, 
                              ompute likelihoods of failure, et
                                                           . However, given that
                                                                              

```

the agent team ommuni ates sele tively about the plans being exe uted, Overseer's planre ognition fa es signi ant un ertainty. Furthermore, Overseer must be able to answer queries on-line, and must therefore work e iently. As dis ussed later, addressing these hallenges has required several novel team-based plan-re ognition te hniques to be developed.

Several onsiderations have led us away from report-based monitoring for this and other Team ore appli ations. First, report-based monitoring requires that agents' ode be modied to ommuni ate the reports needed for monitoring; as monitoring requirements hange
