

## B. Semantics

A set of ground literals is consistent if it does not contain a pair of complementary literals. If a literal belongs to , then we say is true in , and is false in if is in . A set of literals is true in if is contained in . A state is a complete and consistent set of literals that describes the world at a certain time point. Definition 2: Let be a ground action theory in
, be a state, be in , and be an action, where each corresponds to , and (similarly for
). Then, the state resulting from executing is:

after executing $a$ ($a^{\prime}$) in $s$ is given by  $$T(s,a,s^{\prime})=\left\{\begin{array}{ll}p_{i}&ifs^{\prime}=\Phi(a_{i},s)\\ 0&otherwise\end{array}\right.$$ $$O(s,a^{\prime},s^{\prime})=\left\{\begin{array}{ll}p^{\prime}_{i}&ifs^{\prime}=\Phi(a^{\prime}_{i},s)\\ 0&otherwise\end{array}\right.$$  The second-order is $\left\{\begin{array}{ll}\rho^{\prime}_{i}&ifs^{\prime}=\Phi(a^{\prime}_{i},s)\\ 0&otherwise\end{array}\right.$
 
Definition 2: Let be a state, and (similarly ) 
be in propositions. Then, the transition probability distribution after executing is given by The reward received in a state after executing is Definition 3: Let be an initial state, be states, and be a policy in . Then, the value function of n-step remaining, 
, of is given by:
Executing sensing or non-sensing action, in causes a transition to a set of states, . Let denotes the set of actions executed in the states respectively. Notice that if is a singleton, i.e., the same action is executed in every state in , then this corresponds to executing an action in a belief state
. Since executing in produces another set of states , then executing causes a transition from a belief state to another belief state.

$\begin{array}{ccccc}\vdots&1&\vdots\\ \vdots&1&\vdots\\ \vdots&1&\vdots\\ \vdots&1&\vdots\\ \\ 0.85&\vdots\\ 0.15&\vdots\\ 0.85&\vdots\\ 0.15&\vdots\end{array}$  . 

For finite horizon POMDP, a policy can be represented as a set of ordered pairs, starting from the initial belief state (the set of initial states in ), as where for represents a belief state (a set of states) 
resulting from executing in . This set representation of finite horizon policies in POMDP leads to view a policy as a set of trajectories, where each trajectory is where is an initial state in and for all and , such that for any
. Let be a policy for a finite horizon POMDP and be the set of trajectories representation of , given the trajectory view of , the value 

function of can be now described as: where
Thus, the optimal policy , the maximum value function among all policies, is given by .

## V. Reinforcement Learning In Nhplp

This section uses NHPLP to solve reinforcement learning problems, by encoding an action theory, in
, into an np-program, . The probabilistic answer sets of correspond to valid trajectories in with associated value function. The np-program encoding of an action theory in follows related encoding described in 
[30], [31], [37]. We assume that the length of the optimal policy that we are looking for is known and finite. We use the following predicates: for literal holds at time moment for action executes at time , 
for a state of the world at time , 
for the reward received at time is , for the value function of a state at time is , and for the discount factor . If an atom appears in an np-rule in with no annotation it is assumed to be associated with the annotation 1. We use to denote for is a predicate and . Let be the np-program encoding , where is the set of the following np-rules.

Each fluent is encoded as a fact of the form
. Fluent literals are encoded as To specify that fluents and are contrary literals, we use the following np-rules.