was actually requesting  An example TASKFAILURE dialogue is given in Figure HMIHY
interpreted utterance U  as a request to make a third party call e g  to bill it to my home phone  HMIHY then asked the caller for the information it needed to carry out this task the caller complied  and the system completed the call

![0_image_0.png](0_image_0.png)

## 3. Experimental Design

Our experiments apply the machine learning program RIPPER
Cohen to automatically classify the dialogues as problematic or successful  RIPPER is a fast and efficient rule learning system described in more detail in   Cohen we describe it briefly here for completeness RIPPER is based on the incremental reduced error pruning IREP  algorithm described in  Furnkranz Widmer RIPPER improves on IREP
with an information gain metric to guide rule pruning and a Minimum Description Length or MDL based heuristic for determining how many rules should be learned   see  Cohen for more details   Like other learners  RIPPER takes as input the names of a set of classes to be learned  the names and ranges of values of a fixed set of features  and training data specifying the class and feature values for each example in a training set  Its output is a classification model for predicting the class of future examples  expressed as an ordered set of if then rules Although any one of a number of learners could be applied to this problem  we had a nmber of reasons for choosing RIPPER  First  it was important to be able to integrate the results of applying the learner back into the HMIHY spoken dialogue system  Previous work ss are easy for people to understand   Catlett Cohen making it easier to integrate the learned rules into the HMIHY system  Second  RIPPER supports continuous  symbolic and textual bag  set  features  Cohen while other learners  such as  Classification and Regression trees  CART   Brieman  Friedman  Olshen do not support Stone textual bag features There are several textual features in this dataset that prove useful in classifying the dialogues  One of the features that we wished to use was the string representing the recognizer s hypothesis  This is supported in RIPPER because there is no a priori limitation on the size of the set The usefulness of the textual features is exemplified in Finally  previous work in which we had applied other learners to the auto-SLU-
Section