

![0_image_0.png](0_image_0.png)

but in a range of domains, especially those in which wait costs are neither negligible nor accruing too fast.

Thus, we have shown that the MDP produces strategies and that Friday follows these strategies in practice. Moreover, the strategies followed are the ones predicted by the model.

Of practical use, when we followed a prediction of the model, i.e., that an MDP was not required for auctions, we were able to substantially reduce the complexity of one part of the stem. Finally, we showed that the need for strategies was not specifically a phenomenon of the E-Elves domain.

## Mdp Experiments 5.4

Experience using the MDP approach to AA in the E-Elves indicates that it is effective at making reasonable AA decisions. However, in order to determine whether MDPs are a generally useful tool for AA reasoning, more systematic experiments are required. In this section, we present such systematic experiments to determine important properties of MDPs for AA. The MDP reward function is designed to result in the optimal strategy being followed in each state.

In each of the experiments, we vary one of the λ parameters that are the weights of the different factors in Equation 10. The MDP is instantiated with each of a range of values for the parameter and a policy produced for ea value. In ea case, the total policy is defined over 2800 states. The policy is analyzed to determine some basic properties of that policy. In particular, we counted the number of states in which the policy specifies to ask, to delay, to say the user is attending and to say the user is not attending. The statistics show broadly how the policy changes as the parameters change, e.g., whether Friday gives up autonomy more or less when the cost of a coordination change is increased. The first aim of the experiments is to simply confirm that policies change in the desired and expected way when parameters in the reward function are changed.  For instance, if Friday's expected decision quality is increased, there should be more states where it makes an autonomous