
## 3.1 Dynamic Programming Approaches

T π s, t 18 t < t < s t-stage-to-go value functions Vi Vo s L V s s'|π s, t , s V t − 1 s' } s π s,t s' ES    T optimal   VT s ≥ VT s s E  S 7T VT principle of optimality f th
S A s' | s,
2 s'|  V s s aea l
3.1.1 Value Integration  $$value\ \ iteration\ a\ \ algorithm$$  V  $$V_{0}^{*}$$  $$\pi\ \ s,t$$
t

  **Example 3.1**: $\mathbb{C}$  \begin{tabular}{c c c c c} _CR RHC_ & _RHM_ & _G$\star$t$C$_ & _PUM_ & _RHC_ \\ _RHM_ & _RHM_ & _D$\circ$t$C$_ & _D$\circ$t$M$_ & _RHM_ \\  & _RHM_ & _D$\circ$t$C$_ & _CR_ & _RHC_ \\  & _CR_ & & & & \\ \end{tabular}  
18. Recall that for FOMDPs other aspects of history are not relevant.