  Surface solar irradiance (SSI) plays a crucial role in tackling climate
change - as an abundant, non-fossil energy source, exploited primarily via
photovoltaic (PV) energy production. With the growing contribution of SSI to
total energy production, the stability of the latter is challenged by the
intermittent character of the former, arising primarily from cloud effects.
Mitigating this stability challenge requires accurate, uncertainty-aware, near
real-time, regional-scale SSI forecasts with lead times of minutes to a few
hours, enabling robust real-time energy grid management. State-of-the-art
nowcasting methods typically meet only some of these requirements. Here we
present SHADECast, a deep generative diffusion model for the probabilistic
spatiotemporal nowcasting of SSI, conditioned on deterministic aspects of cloud
evolution to guide the probabilistic ensemble forecast, and based on near
real-time satellite data. We demonstrate that SHADECast provides improved
forecast quality, reliability, and accuracy in different weather scenarios. Our
model produces realistic and spatiotemporally consistent predictions
outperforming the state of the art by 15% in the continuous ranked probability
score (CRPS) over different regions up to 512 km x 512 km with lead times of
15-120 min. Conditioning the ensemble generation on deterministic forecasts
improves reliability and performance by more than 7% on CRPS. Our approach
empowers grid operators and energy traders to make informed decisions, ensuring
stability and facilitating the seamless integration of PV energy across
multiple locations simultaneously.
