Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Dynamic Programming Principles for Mean-Field Controls with Learning   Dynamic programming principle (DPP) is fundamental for control and
optimization, including Markov decision problems (MDPs), reinforcement learning
(RL), and more recently mean-field controls (MFCs). However, in the learning
framework of MFCs, DPP has not been rigorously established, despite its
critical importance for algorithm designs. In this paper, we first present a
simple example in MFCs with learning where DPP fails with a mis-specified Q
function; and then propose the correct form of Q function in an appropriate
space for MFCs with learning. This particular form of Q function is different
from the classical one and is called the IQ function. In the special case when
the transition probability and the reward are independent of the mean-field
information, it integrates the classical Q function for single-agent RL over
the state-action distribution. In other words, MFCs with learning can be viewed
as lifting the classical RLs by replacing the state-action space with its
probability distribution space. This identification of the IQ function enables
us to establish precisely the DPP in the learning framework of MFCs. Finally,
we illustrate through numerical experiments the time consistency of this IQ
function.

22
The categories of this paper are: chao-dyn
18
The categories of this paper are: econ
31
The categories of this paper are: ao-sci
26
The categories of this paper are: bayes-an
3
The categories of this paper are: physics
6
The categories of this paper are: astro-ph
15
The categories of this paper are: nucl-ex
36
The categories of this paper are: cmp-lg
19
The categories of this paper are: eess
16
The categories of this paper are: stat
37
The categories of this paper are: supr-con
23
The categories of this paper are: solv-int
5
The categories of this paper are: gr-qc
25
The categories of this paper are: patt-sol
20
The categories of this paper are: acc-phys
4
The categories of this paper are: cond-mat