  Reinforcement learning (RL) has garnered significant attention for developing
decision-making agents that aim to maximize rewards, specified by an external
supervisor, within fully observable environments. However, many real-world
problems involve partial observations, formulated as partially observable
Markov decision processes (POMDPs). Previous studies have tackled RL in POMDPs
by either incorporating the memory of past actions and observations or by
inferring the true state of the environment from observed data. However,
aggregating observed data over time becomes impractical in continuous spaces.
Moreover, inference-based RL approaches often require many samples to perform
well, as they focus solely on reward maximization and neglect uncertainty in
the inferred state. Active inference (AIF) is a framework formulated in POMDPs
and directs agents to select actions by minimizing a function called expected
free energy (EFE). This supplies reward-maximizing (exploitative) behaviour, as
in RL, with information-seeking (exploratory) behaviour. Despite this
exploratory behaviour of AIF, its usage is limited to discrete spaces due to
the computational challenges associated with EFE. In this paper, we propose a
unified principle that establishes a theoretical connection between AIF and RL,
enabling seamless integration of these two approaches and overcoming their
aforementioned limitations in continuous space POMDP settings. We substantiate
our findings with theoretical analysis, providing novel perspectives for
utilizing AIF in the design of artificial agents. Experimental results
demonstrate the superior learning capabilities of our method in solving
continuous space partially observable tasks. Notably, our approach harnesses
information-seeking exploration, enabling it to effectively solve reward-free
problems and rendering explicit task reward design by an external supervisor
optional.
