

![0_image_0.png](0_image_0.png)

Theorem 10 Let CF5M be an FSM controller.  Let CDR and CLA be the direct and the on e-step-lookahead controllers constructed based on CFSM. Then VCrsm(b) ≤ VCDR(b) and VCFSM(b) ≤ VCLA(b) hold for all belief states b ∈ I.

Though we can prove that both the direct controller and the lookahead controller are always better than the underlying FSM controller (see Appendix for the full proof of the theorem), we cannot show the similar property between the first two controllers for all initial belief states. However, the lookahead approach typically tends to dominate, reflecting the usual trade-off between control quality and response time. We illustrate this trade-off on our running Maze20 example and a collection of | A | one-action policies, each generating a sequence of the same action.  Control quality and response time results are shown in Figure 16. We see that the controller based on the FSM is the fastest of the three, but it is also the worst in terms of control quality.  On the other hand, the direct controller is slower (it needs to update belief states in every step) but delivers better control.  Finally, the lookahead controller is the slowest and has the best control performance.

## 4.4.2 Selecting The Fsm Model

The quality of a fixed-strategy approximation depends strongly on the FSM model used. The model can be provided a priori or constructed automatically.  Techniques for automatic construction of FSM policies correspond to a search problem in which either the complete or a restricted space of policies is examined to find the optimal or the near-optimal policy for such a space. The search process is equivalent to policy approximations or policy-iteration techniques discussed earlier in Sections 2.6 and 3.2.