  There is an increasing need for automated support for humans monitoring the
activity of distributed teams of cooperating agents, both human and machine. We
characterize the domain-independent challenges posed by this problem, and
describe how properties of domains influence the challenges and their
solutions. We will concentrate on dynamic, data-rich domains where humans are
ultimately responsible for team behavior. Thus, the automated aid should
interactively support effective and timely decision making by the human. We
present a domain-independent categorization of the types of alerts a plan-based
monitoring system might issue to a user, where each type generally requires
different monitoring techniques. We describe a monitoring framework for
integrating many domain-specific and task-specific monitoring techniques and
then using the concept of value of an alert to avoid operator overload. We use
this framework to describe an execution monitoring approach we have used to
implement Execution Assistants (EAs) in two different dynamic, data-rich,
real-world domains to assist a human in monitoring team behavior. One domain
(Army small unit operations) has hundreds of mobile, geographically distributed
agents, a combination of humans, robots, and vehicles. The other domain (teams
of unmanned ground and air vehicles) has a handful of cooperating robots. Both
domains involve unpredictable adversaries in the vicinity. Our approach
customizes monitoring behavior for each specific task, plan, and situation, as
well as for user preferences. Our EAs alert the human controller when reported
events threaten plan execution or physically threaten team members. Alerts were
generated in a timely manner without inundating the user with too many alerts
(less than 10 percent of alerts are unwanted, as judged by domain experts).
