Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Offline Meta-Reinforcement Learning with Advantage Weighting   This paper introduces the offline meta-reinforcement learning (offline
meta-RL) problem setting and proposes an algorithm that performs well in this
setting. Offline meta-RL is analogous to the widely successful supervised
learning strategy of pre-training a model on a large batch of fixed,
pre-collected data (possibly from various tasks) and fine-tuning the model to a
new task with relatively little data. That is, in offline meta-RL, we
meta-train on fixed, pre-collected data from several tasks in order to adapt to
a new task with a very small amount (less than 5 trajectories) of data from the
new task. By nature of being offline, algorithms for offline meta-RL can
utilize the largest possible pool of training data available and eliminate
potentially unsafe or costly data collection during meta-training. This setting
inherits the challenges of offline RL, but it differs significantly because
offline RL does not generally consider a) transfer to new tasks or b) limited
data from the test task, both of which we face in offline meta-RL. Targeting
the offline meta-RL setting, we propose Meta-Actor Critic with Advantage
Weighting (MACAW), an optimization-based meta-learning algorithm that uses
simple, supervised regression objectives for both the inner and outer loop of
meta-training. On offline variants of common meta-RL benchmarks, we empirically
find that this approach enables fully offline meta-reinforcement learning and
achieves notable gains over prior methods.

10
The categories of this paper are: q-bio
30
The categories of this paper are: q-alg
8
The categories of this paper are: hep-ex
13
The categories of this paper are: quant-ph
9
The categories of this paper are: nlin
2
The categories of this paper are: math
17
The categories of this paper are: q-fin
11
The categories of this paper are: hep-lat
33
The categories of this paper are: plasm-ph
24
The categories of this paper are: dg-ga
29
The categories of this paper are: funct-an
28
The categories of this paper are: alg-geom
12
The categories of this paper are: nucl-th
3
The categories of this paper are: physics
18
The categories of this paper are: econ
14
The categories of this paper are: math-ph