SolarSys: Scalable Hierarchical Coordination for Distributed Solar Energy
SolarSys is a novel Hierarchical Multi-Agent Reinforcement Learning (HRL) system designed to manage energy storage and peer-to-peer (P2P) trading across large communities of solar-equipped residences. This repository contains the full source code for the SolarSys system, including the trained policies, the custom Gym environment, and the hierarchical diffusion model used for data generation.
System Architecture
The core of SolarSys is a two-level decision hierarchy:
- Low-Level (Intra-Cluster): Individual households use a MAPPO agent to make instantaneous decisions (charge, discharge, local P2P trade, grid trade) based on local meter readings and price signals.
- High-Level (Inter-Cluster): Cluster Managers use a Mean-Field policy to coordinate bulk energy transfers between clusters, ensuring the overall system remains balanced against grid constraints.
Data Generation Framework
To enable large-scale simulation with realistic temporal dynamics, SolarSys includes a Hierarchical Diffusion Model for generating synthetic, long-duration energy profiles that maintain both long-term (seasonal/monthly) and short-term (daily/hourly) characteristics.
- Model: Hierarchical Diffusion U-Net
- Input: Location based housing dataset(eg. Ausgrid dataset, newyork dataset)
- Output: High-resolution time series for Grid Usage and Solar Generation (kWh).
To ensure all necessary libraries are available, install dependencies using the provided requirements.txt file:
pip install -r requirements.txt
---
## Repository Structure
The project is organized into core modules and data folders.
```tree
SolarSys_Hugging_face/
├── assets/
├── requirements.txt # Project Dependencies
├── Dataset/ # Energy Data (Raw, Processed, and Synthetic)
│ ├── Data/
│ │ ├── Ausgrid_processed_data.zip # Processed data (Australia)
│ │ ├── Ausgrid_raw_data/
│ │ └── Pecan_street_processed_data.zip # Processed data (US)
│ └── Generated_Data.zip # Example synthetic data (2000 agents)
├── Data_generation_tool_kit/ # Hierarchical Diffusion Model Code
│ ├── Hier_diffusion_energy/
│ │ ├── hierarchial_diffusion_model.py # H-Diffusion U-Net Architecture
│ │ └── global_scaler.gz # Global normalization scaler, please delete and rerun if had problem
│ ├── dataloader.py #
│ ├── train.py # Diffusion model training script
│ └── generate.py # Script to generate long-term sequences
├── Models/ # Trained RL Policies for trial, please retrain your model for best results
│ ├── 100agents_10size/
│ │ └── models/
│ │ ├── inter_ep10000.pth # High-Level (Inter-Cluster) policy
│ │ └── low_clusterX_ep10000.pth # Low-Level (Intra-Cluster) policies
│ └── (Additional scaling folders) # e.g., 5agents_5size/, 1000agents_20size/, etc.
├── SolarSys/ # SolarSys Implementation (MAPPO + MeanField)
│ ├── Environment/
│ │ ├── cluster_env_wrapper.py # Vectorized environment wrapper
│ │ └── solar_sys_environment.py # Core Gym environment definition
│ ├── mappo/ # MAPPO policy definition (Intra-Cluster)
│ │ └── trainer/
│ │ └── mappo.py # MAPPO algorithm
│ ├── meanfield/ # MeanField policy definition (Inter-Cluster)
│ │ └── trainer/
│ │ └── meanfield.py # MeanField algorithm core
│ ├── cluster.py # Inter-Cluster Coordinator logic
│ ├── training_freezing.py # Main SolarSys HRL training script
│ └── cluster_evaluation.py # Evaluation script for SolarSys
└── Other_algorithms/ # Baselines
├── HC_MAPPO/ # Hierarchical Cluster MAPPO
│ ├── Environment/
│ ├── HC_MAPPO_train.py # Training script for HC-MAPPO
│ └── HC_MAPPO_evaluation.py # Evaluation script for HC-MAPPO
└── Flat_System/ # Flat Baselines
├── maddpg/ # Multi-Agent DDPG
│ ├── maddpg_train.py
│ └── maddpg_evaluation.py
├── mappo/ # Flat MAPPO
│ ├── mappo_train.py
│ └── mappo_evaluation.py
├── meanfield/ # Flat MeanField Actor-Critic (MFAC)
│ ├── meanfield_train.py
│ └── meanfield_evaluation.py
├── PG/ # Policy Gradient (PG)
│ ├── pg_train.py
│ └── pg_evaluation.py
└── solar_sys_environment.py # Copy of the flat environment for baselines