As discussed in [1] there are other representations for expressing user preferences, such as trajectory constraints expressed in linear temporal logic. It will be interesting to explore methods for learning preferences in those representations too, and to see to what extent typical user preferences are naturally expressible in
(p)HTNs or such alternatives.

## 6. Conclusion

Despite significant interest in learning in the context of planning, most prior work focused only on learning domain physics or search control. In this paper, we expanded this scope by learning user preferences concerning plans. We developed a framework for learning probabilistic HTNs from a set of example plans, drawing from the literature on probabilistic grammar induction. Assuming the input distribution is in fact sampled from a pHTN, we demonstrated that the approach finds a pHTN generating a similar distribution. It is, however, a stretch to imagine that we can sample directly from such a distribution - chiefly because observed behavior arises from a complex interaction between preferences and physics.

We demonstrate a technique overcoming the effect of such feasibility constraints, by reasoning about the available alternatives to the observed user behavior. The technique is to rescale the distribution to fit the assumptions of the baseline pHTN learner. We evaluate the approach, and demonstrate both that the original learner is easily confounded by constraints placed upon the preference distribution, and that rescaling is effective at reversing this effect. We discuss several remaining important directions for future work to address. Of these, the most directly relevant technical pursuit is learning parameterized pHTNs, or more generally, learning conditional preferences. Fully integrating an automated planner with the learner, thereby using the learned knowledge, and running (costly...)
user studies are also very important pursuits. In the end, we describe an effective approach to automatically learning a model of a user's preferences from observations of only their behavior.

Acknowledgments: Kambhampati's research is supported in part by ONR grants N00014-09-1-0017 and N00014-07-1-1049, and the DARPA Integrated Learning Program (through a sub-contract from Lockheed Martin).

## References

[1] J. A. Baier, S. A. McIlraith, Planning with preferences, AI Magazine 29 (4)
(2008) 25–36.