# Asimovian Adaptive Agents

## Dia Na  F.  Gordon

GORDON QAIC. N RL. NAVY. MIL
Navy Center for Applied Research in Artificial Intelligence Naval Research Laboratory, Code 5515 Washington, D.C. 20375-5337 USA

## Abstract

The goal of this research is to develop agents that are adaptive and predictable and timely. At first blush, these three requirements seem contradictory. For example, adaptation risks introducing undesirable side effects, thereby making agents' behavior less predictable. Furthermore, although formal verification can assist in ensuring behavioral predictability, it is known to be time-consuming.

Our solution to the challenge of satisfying all three requirements is the following.  Agents have finite-state automaton plans, which are adapted online via evolutionary learning (perturbation) operators. To ensure that critical behavioral constraints are always satisfied, agents' plans are first formally verified. They are then reverified after every adaptation.

If reverification concludes that constraints are violated, the plans are repaired. The main objective of this paper is to improve the efficiency of reverification after learning, so that agents have a sufficiently rapid response time. We present two solutions: positive results that certain learning operators are a priori guaranteed to preserve useful classes of behavioral assurance constraints (which implies that no reverification is needed for these operators), and efficient incremental reverification algorithms for those learning operators that have negative a priori results.

## 1. Introduction

Agents are becoming increasingly prevalent and effective Robots and softbots  working individually or in concert  can relieve people of a great deal of labor  intensive tedium in their jobs as well as in their day to day lives  Designers can furnish agents with plans to perform desired tasks  Nevertheless a designer cannot possibly foresee all circumstances that will be encountered by the agent Therefore in addition to supplying an agent with plans it is essential to also enable the agent to learn and modify its plans to adapt to unforeseen circumstances The introduction of learning however often makes the agent s behavior significantly harder to predict 1 The goal of this research is to verify the behavior of adaptive agents In particular our objective is to develop efficient methods for determining whether the behavior of learning agents remains within the bounds of prespecified constraints   called properties" after learning This includes verifying that properties are preserved for single adaptive agents as well as verifying that global properties are preserved for multiagent systems in which one or more agents may adapt An example of a property is Asimov s First Law Asimov This law which has also been studied by Weld and Etzioni states that an agent may not harm a