is high, the learning will involve improving the estimate of h(α) based on the feedback about the specific plan that the user selects from the set returned by the system. This learning phase is in principle well connected to the Bayesian parameter estimation approach in the sense that the whole distribution of parameter vector, h(α), is updated after receiving feedback from the user, taking into account the current distribution of all models (starting from a prior, for instance the uniform distribution). Although such interactive learning framework has been discussed previously, as in Chajewska et al. (2001), the set of user's decisions in this work is assumed to be given, whereas in planning scenarios the cost of plan synthesis should be incoporated into the our interactive framework, and the problem of presenting plan sets to the user needs also to be considered. Recent work by Li et al. (2009) considered learning user's preferences in planning, but restricting to preference models that can be represented with hierachical task networks.

Acknowledgements: We thank Menkes van den Briel for drawing our attention to ICP measure initially. Tuan Nguyen's research is supported by a Science Foundation of Arizona fellowship. Kambhampati's research is supported in part by an IBM Faculty Award, the NSF grant IIS2013308139, ONR grants N00014-09-1-0017, N00014-07-1-10 ˘ 49, N000140610058, and by a Lockheed Martin subcontract TT0687680 to ASU as part of the DARPA Integrated Learning program.

## References

Baier, J., McIlraith, S., 2009. Planning with preferences. AI Magazine 29 (4), 25.

Barrett, L., Narayanan, S., 2008. Learning all optimal policies with multiple criteria. In: Proceedings of the 25th international conference on Machine learning. ACM, pp. 41–47.

Bienvenu, M., Fritz, C., McIlraith, S., 2006. Planning with qualitative temporal preferences. In: Proceedings of the 10th International Conference on Knowledge Representation and Reasoning (KR). pp. 134–144.

Birkhoff, G., 1948. Lattice Theory, volume XXV of American Mathematical Society Colloquium Publications. American Mathematical Society, New York, 2nd (revised) edition.

Boddy, M., Gohde, J., Haigh, T., Harp, S., 2005. Course of action generation for cyber security using classical planning.

Biundo et al.(Biundo, Myers, & Rajan, 2005), 12–21.

Boutilier, C., Brafman, R., Domshlak, C., Hoos, H., Poole, D., 2004. CP-nets: A tool for representing and reasoning with conditional ceteris paribus preference statements. Journal of Artificial Intelligence Research 21 (1), 135–191.

Boutilier, C., Dean, T., Hanks, S., 1999. Decision-theoretic planning: Structural assumptions and computational leverage.

Journal of Artificial Intelligence Research 11 (1), 94.

Boutilier, C., Regan, K., Viappiani, P., 2010. Simultaneous Elicitation of Preference Features and Utility.

Brafman, R., Domshlak, C., 2009. Preference handling–An introductory tutorial. AI magazine 30 (1), 58–86. Brafman, R. I., Chernyavsky, Y., 2005. Planning with goal preferences and constraints. In: International Conference on Automated Planning and Scheduling. pp. 182–191.

Bryce, D., Cushing, W., Kambhampati, S., 2007. Model-lite planning: Diverse multi-option plans & dynamic objective functions. In: ICAPS 2007 Workshop on Planning and Plan Execution for Real World Systems.

Carlyle, W. M., Fowler, J. W., E. S. Gel, B. K., 2003. Quantitative comparison of approximate solution sets for bi-criteria optimization problems. Decision Sciences 34 (1).

Chafle, G., Dasgupta, K., Kumar, A., Mittal, S., Srivastava, B., 2006. Adaptation inWeb Service Composition and Execution. In: International Conference on Web Services, 2006. ICWS'06. pp. 549–557.

Chajewska, U., Koller, D., Ormoneit, D., 2001. Learning an agent's utility function by observing behavior. In: Proceedings of the Eighteenth International Conference on Machine Learning. pp. 35–42.

Chajewska, U., Koller, D., Parr, R., 2000. Making rational decisions using adaptive utility elicitation. In: Proceedings of the National Conference on Artificial Intelligence. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT
Press; 1999, pp. 363–369.

Delage, E., Mannor, S., 2007. Percentile optimization in uncertain Markov decision processes with application to efficient exploration. In: Proceedings of the 24th international conference on Machine learning. ACM, p. 232.

Do, M., Kambhampati, S., 2001. Planning as constraint satisfaction: Solving the planning graph by compiling it into CSP* 1. Artificial Intelligence 132 (2), 151–182.