Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Scalable methods for computing state similarity in deterministic Markov
  Decision Processes   We present new algorithms for computing and approximating bisimulation
metrics in Markov Decision Processes (MDPs). Bisimulation metrics are an
elegant formalism that capture behavioral equivalence between states and
provide strong theoretical guarantees on differences in optimal behaviour.
Unfortunately, their computation is expensive and requires a tabular
representation of the states, which has thus far rendered them impractical for
large problems. In this paper we present a new version of the metric that is
tied to a behavior policy in an MDP, along with an analysis of its theoretical
properties. We then present two new algorithms for approximating bisimulation
metrics in large, deterministic MDPs. The first does so via sampling and is
guaranteed to converge to the true metric. The second is a differentiable loss
which allows us to learn an approximation even for continuous state MDPs, which
prior to this work had not been possible.

37
The categories of this paper are: supr-con
30
The categories of this paper are: q-alg
8
The categories of this paper are: hep-ex
29
The categories of this paper are: funct-an
9
The categories of this paper are: nlin
10
The categories of this paper are: q-bio
14
The categories of this paper are: math-ph
31
The categories of this paper are: ao-sci
32
The categories of this paper are: atom-ph
23
The categories of this paper are: solv-int
28
The categories of this paper are: alg-geom
3
The categories of this paper are: physics
36
The categories of this paper are: cmp-lg
26
The categories of this paper are: bayes-an
21
The categories of this paper are: adap-org
13
The categories of this paper are: quant-ph