  Training networks consisting of biophysically accurate neuron models could
allow for new insights into how brain circuits can organize and solve tasks. We
begin by analyzing the extent to which the central algorithm for neural network
learning -- stochastic gradient descent through backpropagation (BP) -- can be
used to train such networks. We find that properties of biophysically based
neural network models needed for accurate modelling such as stiffness, high
nonlinearity and long evaluation timeframes relative to spike times makes BP
unstable and divergent in a variety of cases. To address these instabilities
and inspired by recent work, we investigate the use of "gradient-estimating"
evolutionary algorithms (EAs) for training biophysically based neural networks.
We find that EAs have several advantages making them desirable over direct BP,
including being forward-pass only, robust to noisy and rigid losses, allowing
for discrete loss formulations, and potentially facilitating a more global
exploration of parameters. We apply our method to train a recurrent network of
Morris-Lecar neuron models on a stimulus integration and working memory task,
and show how it can succeed in cases where direct BP is inapplicable. To expand
on the viability of EAs in general, we apply them to a general neural ODE
problem and a stiff neural ODE benchmark and find again that EAs can
out-perform direct BP here, especially for the over-parameterized regime. Our
findings suggest that biophysical neurons could provide useful benchmarks for
testing the limits of BP-adjacent methods, and demonstrate the viability of EAs
for training networks with complex components.
