  Partial differential equations (PDEs) are important tools to model physical
systems and including them into machine learning models is an important way of
incorporating physical knowledge. Given any system of linear PDEs with constant
coefficients, we propose a family of Gaussian process (GP) priors, which we
call EPGP, such that all realizations are exact solutions of this system. We
apply the Ehrenpreis-Palamodov fundamental principle, which works as a
non-linear Fourier transform, to construct GP kernels mirroring standard
spectral methods for GPs. Our approach can infer probable solutions of linear
PDE systems from any data such as noisy measurements, or pointwise defined
initial and boundary conditions. Constructing EPGP-priors is algorithmic,
generally applicable, and comes with a sparse version (S-EPGP) that learns the
relevant spectral frequencies and works better for big data sets. We
demonstrate our approach on three families of systems of PDEs, the heat
equation, wave equation, and Maxwell's equations, where we improve upon the
state of the art in computation time and precision, in some experiments by
several orders of magnitude.
