The automatic results given in row   are significantly higher by a paired t test than The baseline for all three sections of the dialogue  df t df p df p t p Rows   and   show accuracy improvements gained by the addition of hand labelled features These rows give a topline against which to compare the results in rows Results using all the automatic features plus the hand labelled SLU-success and are given in row    In these experiments  the hand labelled SLU-success feature is used for training and testing Comparing this result with the second row shows that if one had a perfect predictor of auto-SLU-success in the training and the test set then this feature would increase accuracy by      for Exchange    from to by for Exchanges and by       for the whole dialogue from           to to The the content of the content to t p df t df t P
Comparing the result in row   with the result in row   shows that the auto-SLU-success predictor that we have trained can improve performance  but could possibly help more with different training methods  Ideally the result in row    for automatic features plus auto¬
SLU-success should fall between the figures in rows   and    and be closer to the results With Exchanges       adding auto-SLU-success results in an increase of in row which is not significant  compare rows   and     For Exchange   only  RIPPER does not use the auto-SLU-success feature in its ruleset and does not yield an improvement over the system trained only on the automatic features The system trained on the whole dialogue with automatic features plus auto-SLU-success also does not yield an improvement over the system trained without auto-SLU-success

## 6.1.1  Task-Independent Features

Rows and give the results using the AUTO, TASK-INDEPT feature set described in Figure without and with the auto-SLU-success feature  respectively  These results are signifi The cantly above the baseline using a paired t test  with Exchanges giving an increase of using TASK-INDEPT features with auto-SLU-success  By d f p comparing rows   and    one observes an increase in the AUTO,  TASK-INDEP T features set when the feature auto-SLU-success is added using Exchanges         and whole dialogue  The increase for Exchanges       shows a trend  df         t whereas the p increase for the whole dialogue is statistically significant by a paired t test  df p Although  the  TASK-INDEPT feature sets are a subset of those features used in row    it is possible for them to perform better because the TASK-INDEPT features are more general and because RIPPER uses a greedy algorithm to discover its rule sets  For Exchanges the increase from row   to row    both of which use auto-SLU-success  is not significant Comparing rows   and    neither of which use auto-SLU-success  one sees a slight degrada tion in results for the whole dialogue using TASK-INDEPT features  However  the increase to            for Exchanges        is statistically significant   df from rows to from This shows that using auto-SLU-success in combination with the set of t TASK-INDEPT features produces a statistically significant increase in accuracy over a set of automatic features that does not include this feature