  Given a training set, a loss function, and a neural network architecture, it
is often taken for granted that optimal network parameters exist, and a common
practice is to apply available optimization algorithms to search for them. In
this work, we show that the existence of an optimal solution is not always
guaranteed, especially in the context of {\em sparse} ReLU neural networks. In
particular, we first show that optimization problems involving deep networks
with certain sparsity patterns do not always have optimal parameters, and that
optimization algorithms may then diverge. Via a new topological relation
between sparse ReLU neural networks and their linear counterparts, we derive --
using existing tools from real algebraic geometry -- an algorithm to verify
that a given sparsity pattern suffers from this issue. Then, the existence of a
global optimum is proved for every concrete optimization problem involving a
shallow sparse ReLU neural network of output dimension one. Overall, the
analysis is based on the investigation of two topological properties of the
space of functions implementable as sparse ReLU neural networks: a best
approximation property, and a closedness property, both in the uniform norm.
This is studied both for (finite) domains corresponding to practical training
on finite training sets, and for more general domains such as the unit cube.
This allows us to provide conditions for the guaranteed existence of an optimum
given a sparsity pattern. The results apply not only to several sparsity
patterns proposed in recent works on network pruning/sparsification, but also
to classical dense neural networks, including architectures not covered by
existing results.
