  Heat waves are projected to increase in frequency and severity with global
warming. Improved warning systems would help reduce the associated loss of
lives, wildfires, power disruptions, and reduction in crop yields. In this
work, we explore the potential for deep learning systems trained on historical
data to forecast extreme heat on short, medium and subseasonal timescales. To
this purpose, we train a set of neural weather models (NWMs) with convolutional
architectures to forecast surface temperature anomalies globally, 1 to 28 days
ahead, at $\sim200~\mathrm{km}$ resolution and on the cubed sphere. The NWMs
are trained using the ERA5 reanalysis product and a set of candidate loss
functions, including the mean squared error and exponential losses targeting
extremes. We find that training models to minimize custom losses tailored to
emphasize extremes leads to significant skill improvements in the heat wave
prediction task, compared to NWMs trained on the mean squared error loss. This
improvement is accomplished with almost no skill reduction in the general
temperature prediction task, and it can be efficiently realized through
transfer learning, by re-training NWMs with the custom losses for a few epochs.
In addition, we find that the use of a symmetric exponential loss reduces the
smoothing of NWM forecasts with lead time. Our best NWM is able to outperform
persistence in a regressive sense for all lead times and temperature anomaly
thresholds considered, and shows positive regressive skill compared to the
ECMWF subseasonal-to-seasonal control forecast after two weeks.
