  The decarbonization of buildings presents new challenges for the reliability
of the electrical grid as a result of the intermittency of renewable energy
sources and increase in grid load brought about by end-use electrification. To
restore reliability, grid-interactive efficient buildings can provide
flexibility services to the grid through demand response. Residential demand
response programs are hindered by the need for manual intervention by
customers. To maximize the energy flexibility potential of residential
buildings, an advanced control architecture is needed. Reinforcement learning
is well-suited for the control of flexible resources as it is able to adapt to
unique building characteristics compared to expert systems. Yet, factors
hindering the adoption of RL in real-world applications include its large data
requirements for training, control security and generalizability. Here we
address these challenges by proposing the MERLIN framework and using a digital
twin of a real-world 17-building grid-interactive residential community in
CityLearn. We show that 1) independent RL-controllers for batteries improve
building and district level KPIs compared to a reference RBC by tailoring their
policies to individual buildings, 2) despite unique occupant behaviours,
transferring the RL policy of any one of the buildings to other buildings
provides comparable performance while reducing the cost of training, 3)
training RL-controllers on limited temporal data that does not capture full
seasonality in occupant behaviour has little effect on performance. Although,
the zero-net-energy (ZNE) condition of the buildings could be maintained or
worsened as a result of controlled batteries, KPIs that are typically improved
by ZNE condition (electricity price and carbon emissions) are further improved
when the batteries are managed by an advanced controller.
