  Human immunodeficiency virus (HIV) is a major public health concern in the
United States, with about 1.2 million people living with HIV and 35,000 newly
infected each year. There are considerable geographical disparities in HIV
burden and care access across the U.S. The 2019 Ending the HIV Epidemic (EHE)
initiative aims to reduce new infections by 90% by 2030, by improving coverage
of diagnoses, treatment, and prevention interventions and prioritizing
jurisdictions with high HIV prevalence. Identifying optimal scale-up of
intervention combinations will help inform resource allocation. Existing HIV
decision analytic models either evaluate specific cities or the overall
national population, thus overlooking jurisdictional interactions or
differences. In this paper, we propose a multi-agent reinforcement learning
(MARL) model, that enables jurisdiction-specific decision analyses but in an
environment with cross-jurisdictional epidemiological interactions. In
experimental analyses, conducted on jurisdictions within California and
Florida, optimal policies from MARL were significantly different than those
generated from single-agent RL, highlighting the influence of jurisdictional
variations and interactions. By using comprehensive modeling of HIV and
formulations of state space, action space, and reward functions, this work
helps demonstrate the strengths and applicability of MARL for informing public
health policies, and provides a framework for expanding to the national-level
to inform the EHE.
