Problems   Many planning systems were developed to solve a particular type of planning problem or explore a specific type of algorithmic variation   Consequently  one would expect them to perform better on the problems on which and for which they were developed  Even were they not designed for a specific purpose  the test set used during development may have subtly biased the development The community knows that planner performance depends on problem features  but not in general  how  when and  why   Researchers tend to design planners to be general purpose  Consequently  comparisons assume that the performance of a general-purpose planner should not be penalized/biased if executed on a sampling of problems and domains  problem assumption The community also knows that problem representation influences planner performance For example  benchmark problem sets include many versions of Blocksworld problems  de signed by different planner developers These versions vary in their problem representation both minor apparently syntactic changes  e g   how clauses are ordered within operators initial conditions and goals  and whether any information is extraneous  and changes re flecting addition of domain knowledge  e g  what constraints are included and whether variables are typed Consequently comparisons assume that syntactic representational modifications either do not matter or affect each planner equally problem assumption PDDL includes a field  : requirements  for the capabilities required of a planner to solve defined    values for the :requirements field  the base  default re the problem  PDDL
quirement is :strips meaning STRIPS derived add and delete sets for action effects  : adl from Pednault s Action Description Language  requires variable typing  disjunctive pre conditions equality as a built in predicate quantified preconditions and conditional effects in addition the : strips capability  Yet many planners either ignore the : requirements field or reject the problem only if it specifies : adl  ignoring many of the other requirements that could also cause trouble  Thus  comparisons assume that problems in the benchmark set should be solvable by a STRIPS planner unless they require : adl  problem assumption Planners    The wonderful trend of making planners publicly available has led to a dilemma in determining which to use and how to configure them The problem is compounded by the longevity of some of these planner projects  some projects have produced multiple versions Consequently comparisons tend to assume that the latest version of the planner is the best planner assumption These planners may also include parameters For example the blackbox planner allows the user to define a strategy for applying different solution methods  Researchers expect that parameters affect performance  Consequently  comparisons assume that default parameter settings approximate good performance  planner assumption