  When neural networks are used to solve differential equations, they usually
produce solutions in the form of black-box functions that are not directly
mathematically interpretable. We introduce a method for generating symbolic
expressions to solve differential equations while leveraging deep learning
training methods. Unlike existing methods, our system does not require learning
a language model over symbolic mathematics, making it scalable, compact, and
easily adaptable for a variety of tasks and configurations. As part of the
method, we propose a novel neural architecture for learning mathematical
expressions to optimize a customizable objective. The system is designed to
always return a valid symbolic formula, generating a useful approximation when
an exact analytic solution to a differential equation is not or cannot be
found. We demonstrate through examples how our method can be applied on a
number of differential equations, often obtaining symbolic approximations that
are useful or insightful. Furthermore, we show how the system can be
effortlessly generalized to find symbolic solutions to other mathematical
tasks, including integration and functional equations.
