
| Train MI               |          |                 |      |
|------------------------|----------|-----------------|------|
| Measure                | Train SI | | Overall Train | Test |
| Comp                   |          |                 |      |
| User Turns             |          |                 |      |
| System Turns           |          |                 |      |
| Elapsed time  sec      |          |                 |      |
| Mean recognition score |          |                 |      |
| TimeOuts               |          |                 |      |
| TimeOut                |          |                 |      |
| Cancs                  |          |                 |      |
| Canc                   |          |                 |      |
| Help Requests          |          |                 |      |
| Help                   |          |                 |      |
| BargeIns               |          |                 |      |
| BargeIn                |          |                 |      |
| Rejects                |          |                 |      |
| Reject                 |          |                 |      |
| User satisfaction      |          |                 |      |

learned via reinforcement learning  if an appropriate performance function could be deter mined  and described an experiment using genetic algorithms to learn an optimal dialogue strategy  In subsequent work  utilized here  the PARADISE model was proposed as a way to time learn appropriate performance function   Walker et al a In addition related work utilizing Elvis  that varied the reward function  and applied other reinforcement learning algorithms  was carried out by Fromer   Fromer proposed the use of similar techniques in the context of Biermann and Long learning optimal dialogue strategies for a multi modal dialogue tutor  The goal of the tutor was to instruct students taking their first programming class and the tutor interacted with the students by highlighting parts of their code and printing text on the screen telling them what was wrong with their program  Biermann and Long describe a planned experiment in which the system would vary its instructional style  and the system s reward would be the amount of time between the system s instructions and the student s response   This reward function was based on the assumption that a delayed response suggested a greater cognitive load for the student  and that cognitive load should be minimized in an instructional setting Levin and colleagues also proposed treating dialogue systems as Markov Decision Pro cesses and suggested that system designers could determine what an appropriate objective function might be  Levin et al Levin    Pieraccini They carried out a series of experiments in which a simulated user interacted with an implemented spoken dialogue system for travel planning by exchanging messages at the semantic meaning level  They showed that the system could learn strategy oices at the level of database interaction