to lead to significant decreases in user satisfaction  Shriberg  Wade Price Danieli Gerbino Kamm Baggia et al Furthermore  user satisfaction alone fails to reflect the fact that the system will not be successful unless it helps the user complete a task We concluded that the relationship between these measures is both interesting and complex and that a method for deriving an appropriate performance function was a necessary precursor to applying stochastic optimization algorithms to spoken dialogue ssees to the sense a we proposed the PARADISE method for learning a performance function from a corpus of human computer dialogues In this work  we apply the PARADISE model to learn a performance function for ELVIS
which we then use for calculating the utility of the final state of a dialogue in experiments applying reinforcement learning to ELVIS s selection of dialogue strategies  Section scribes the implementation of a version of ELVIS that randomly explores alternate strategies for initiative  for reading messages  and for summarizing email folders   Section    describes the experimental design in which we first use this exploratory version of ELVIS to collect a training corpus of conversations with    human users carrying out a set of three email tasks   Section   describes how we apply reinforcement learning to the corpus of dia logues to optimize ELVIS s dialogue strategy decisions  We then test the optimized policy in an experiment in which six new users interact with ELVIS to complete the same set of tasks and show that the learned policy performs significantly better than the exploratory policy used during the training phase

## 2.   Elvis Spoken Dialogue System

We started the process of designing ELVIS by conducting a Wizard of Oz experiment in which we recorded dialogues with six people accessing their email remotely by talking to a human who was playing the part of the spoken dialogue system The purpose of this experiment was to identify the basic functionality that should be implemented in ELVIS
The analysis of the resulting dialogues suggested that ELVIS needed to support content based access to email messages by specification of the subject or the sender field  verbal summaries of email folders  reading the body of an email message  and requests for help and repetition of messages  Walker et al b Given these requirements  we then implemented ELVIS using a general purpose platform for spoken dialogue systems  Kamm et al The platform consists of a dialogue mm a speech recognizer  an audio server for both voice recordings and text to speech  TTS  an interface between the computer running ELVIS and the telephone network  and modules for specifying the rules for spoken language understanding and application specific functions The speech recognizer is a speaker independent Hidden Markov Model  HMM  system with context dependent phone models for telephone speech and constrained grammars defining the vocabulary that is permitted at any point in a dialogue  Rabiner  Juang Lee The platform supports barge in so that the user can interrupt the system barge in is very important for this application so that the user can interrupt the system when it is reading a long email message The audio server can switch between voice recordings and text to speech  TTS  and integrate voice recordings with TTS The TTS technology is concatenative diphone synthe