  This paper addresses the important need for advanced techniques in
continuously allocating workloads on shared infrastructures in data centers, a
problem arising due to the growing popularity and scale of cloud computing. It
particularly emphasizes the scarcity of research ensuring guaranteed capacity
in capacity reservations during large-scale failures. To tackle these issues,
the paper presents scalable solutions for resource management. It builds on the
prior establishment of capacity reservation in cluster management systems and
the two-level resource allocation problem addressed by the Resource Allowance
System (RAS). Recognizing the limitations of Mixed Integer Linear Programming
(MILP) for server assignment in a dynamic environment, this paper proposes the
use of Deep Reinforcement Learning (DRL), which has been successful in
achieving long-term optimal results for time-varying systems. A novel two-level
design that utilizes a DRL-based algorithm is introduced to solve optimal
server-to-reservation assignment, taking into account of fault tolerance,
server movement minimization, and network affinity requirements due to the
impracticality of directly applying DRL algorithms to large-scale instances
with millions of decision variables. The paper explores the interconnection of
these levels and the benefits of such an approach for achieving long-term
optimal results in the context of large-scale cloud systems. We further show in
the experiment section that our two-level DRL approach outperforms the MIP
solver and heuristic approaches and exhibits significantly reduced computation
time compared to the MIP solver. Specifically, our two-level DRL approach
performs 15% better than the MIP solver on minimizing the overall cost. Also,
it uses only 26 seconds to execute 30 rounds of decision making, while the MIP
solver needs nearly an hour.
