  In many mechanistic medical, biological, physical and engineered
spatiotemporal dynamic models the numerical solution of partial differential
equations (PDEs) can make simulations impractically slow. Biological models
require the simultaneous calculation of the spatial variation of concentration
of dozens of diffusing chemical species. Machine learning surrogates, neural
networks trained to provide approximate solutions to such complicated numerical
problems, can often provide speed-ups of several orders of magnitude compared
to direct calculation. PDE surrogates enable use of larger models than are
possible with direct calculation and can make including such simulations in
real-time or near-real time workflows practical. Creating a surrogate requires
running the direct calculation tens of thousands of times to generate training
data and then training the neural network, both of which are computationally
expensive. We use a Convolutional Neural Network to approximate the stationary
solution to the diffusion equation in the case of two equal-diameter, circular,
constant-value sources located at random positions in a two-dimensional square
domain with absorbing boundary conditions. To improve convergence during
training, we apply a training approach that uses roll-back to reject stochastic
changes to the network that increase the loss function. The trained neural
network approximation is about 1e3 times faster than the direct calculation for
individual replicas. Because different applications will have different
criteria for acceptable approximation accuracy, we discuss a variety of loss
functions and accuracy estimators that can help select the best network for a
particular application.
