problem. The importance of that is reinforcement learning problems in POMDP environment can be now solved as SAT 
problems.

II. SYNTAX AND SEMANTICS OF NHPLP
We introduce a class of NHPLP [34], namely NHPLP
that is sufficient to represent POMDP.

## A. The Language Of Nhplp

 Let be a first-order language with finitely many predicate symbols, constants, and infinitely many variables. 

The Herbrand base of is denoted by . Probabilities are assigned to atoms in as values from . An *annotation*, 
, is either a constant in , a variable (annotation variable) ranging over , or (called annotation function) where is a representation of a computable total function and are annotations. Let . Then we say that iff . A normal probabilistic logic program (*np-program*) in NHPP is a pair where is a finite set of normal probabilistic rules (np-rules) 
and is a mapping , where is a set of disjunctive probabilistic strategies (p-strategies) whose composition functions , are mappings 
. A composition function of a disjunctive p-strategy returns the probability of a disjunction of two events given the probability values of its components An np-rule is an expression of the form where are atoms and are annotations. Intuitively, the meaning of an np-rule is that if for each , the probability of is at least (w.r.t. ) and for each 
, it is not *believable* that the probability of is at least , then the probability of is . The mapping associates to each atom a disjunctive p-strategy that will be employed to combine the probability values obtained from different np-rules having in their heads. An np-program is ground if no variables appear in any of its p-rules.

## B. Probabilistic Answer Set Semantics Of Nhplp

 A probabilistic interpretation (p-interpretation), , is a mapping from to . Let be a ground npprogram, be a p-interpretation, and be an np-rule as above. Then, we say The probabilistic reduct of w.r.t. is an np-program without negation, , where:

$A:\mu\gets A_{1}:\mu_{1},\ldots,A_{n}:\mu_{n},$  $not\ (B_{1}:\beta_{1}),\ldots,not\ (B_{m}:\beta_{m})\in R,$  $\forall(1\leq j\leq m),\ \beta_{j}\not\leq h(B_{j}).$ A probabilistic model 
and A probabilistic model (*pmodel*) of an np-program is a p-interpretation of that satisfies . We say that a p-interpretation of is a probabilistic answer set of if is the minimal p-model of the probabilistic reduct, , of w.r.t. .

## Iii. Partially Observable Markov Decision Processes

We review finite-horizon POMDP [12] with stationary transition functions, stationary bounded reward functions, and stationary policies.

## A. Pomdp Definition

POMDP is a tuple of the form where: is a finite set of states; is the initial state distribution; is a finite set of stochastic actions; is stationary transition function
, where for any 

$=\;\frac{(-1)^{a/b}+1)^{a/b}\sum_{s\in S}\;(-1)^{-s/b}}{Pr(o|a,b)}$  . 
$$V_{n}^{\pi}(b)=\sum_{s\in S}b(s)V_{n}^{\pi}(s).$$
which determines the expected sum of discounted rewards resulting from executing the policy starting from . Because of the agent is unable to completely observe the states of the world and with reliability, it keeps what is called a belief state. An agent's belief state is a probability distribution over the possible world states the agent may think it is in. Therefore, an action causes a transition from a belief state to another belief state. Given is a believe state and is an action, then executing in the belief state results a new belief state , where the probability of a state, , in and the value function of executing a policy in are given by: