  We derive a learning framework to generate routing/pickup policies for a
fleet of autonomous vehicles tasked with servicing stochastically appearing
requests on a city map. We focus on policies that 1) give rise to coordination
amongst the vehicles, thereby reducing wait times for servicing requests, 2)
are non-myopic, and consider a-priori potential future requests, 3) can adapt
to changes in the underlying demand distribution. Specifically, we are
interested in policies that are adaptive to fluctuations of actual demand
conditions in urban environments, such as on-peak vs. off-peak hours. We
achieve this through a combination of (i) an online play algorithm that
improves the performance of an offline-trained policy, and (ii) an offline
approximation scheme that allows for adapting to changes in the underlying
demand model. In particular, we achieve adaptivity of our learned policy to
different demand distributions by quantifying a region of validity using the
q-valid radius of a Wasserstein Ambiguity Set. We propose a mechanism for
switching the originally trained offline approximation when the current demand
is outside the original validity region. In this case, we propose to use an
offline architecture, trained on a historical demand model that is closer to
the current demand in terms of Wasserstein distance. We learn routing and
pickup policies over real taxicab requests in San Francisco with high
variability between on-peak and off-peak hours, demonstrating the ability of
our method to adapt to real fluctuation in demand distributions. Our numerical
results demonstrate that our method outperforms alternative rollout-based
reinforcement learning schemes, as well as other classical methods from
operations research.
