erences, and that our rescaling technique is able to recover preference knowledge despite obfuscation.

## 4.4.1. Setup

Performance. We again take an oracle-based experimental strategy, that is, we imagine a user with a particular ideal pHTN, H∗, representing that user's preferences, and then test the efficacy of the learner at recovering knowledge of preferences based on observations of the imaginary user. More specifically we test the learner's performance in the following game. After training the learner produces Hr; to evaluate the effectiveness of Hr we pick random plan pairs and ask both H∗and Hr to pick the preferred plan. There are three possibilities: Hr agrees with H∗(+1 point), Hr disagrees with H∗(-1 point), and Hr declines to choose
(0 points)9.

The distribution on testing plans is not uniform and will be described below.

The number of plan pairs used for testing is scaled by the size of H∗; 100t pairs are generated, where t is the number of non-primitives. The final performance for one instance of the game is the average number of points earned per testing pair.

Pure guessing, then, would get (in the long-term) 0 performance.

User. As in the prior evaluation we evaluate on 1) randomly generated pHTNs modeling possible users, and on 2) hand-crafted pHTNs modeling our preferences in Logistics and Gold Miner. Training Data. For both randomly generated and hand-crafted users we use the same framework for generating training data. We generate random problems by generating random solution sets in a particular fashion, that is, *we model feasibility* constraints using a particular random distribution on solution sets. We describe this process in detail below, but note that the details are unimportant (we did not try any alternatives to the choices given); what matters is whether or not the process is a reasonable model of the effect of feasibility constraints (insofar as they affect the learning problem).

We begin by constructing a list of plans, P, from 100t samples of H∗, removing duplicates (so **|P| ≤** 100t). Due to duplicate removal, less preferred plans occur later than more preferred plans (on average). We reverse that order, and associate P with (a discrete approximation to) a power-law distribution. Both training and test plans are drawn from this distribution. Then, for each training