Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: The merged-staircase property: a necessary and nearly sufficient
  condition for SGD learning of sparse functions on two-layer neural networks   It is currently known how to characterize functions that neural networks can
learn with SGD for two extremal parameterizations: neural networks in the
linear regime, and neural networks with no structural constraints. However, for
the main parametrization of interest (non-linear but regular networks) no tight
characterization has yet been achieved, despite significant developments.
  We take a step in this direction by considering depth-2 neural networks
trained by SGD in the mean-field regime. We consider functions on binary inputs
that depend on a latent low-dimensional subspace (i.e., small number of
coordinates). This regime is of interest since it is poorly understood how
neural networks routinely tackle high-dimensional datasets and adapt to latent
low-dimensional structure without suffering from the curse of dimensionality.
Accordingly, we study SGD-learnability with $O(d)$ sample complexity in a large
ambient dimension $d$.
  Our main results characterize a hierarchical property, the "merged-staircase
property", that is both necessary and nearly sufficient for learning in this
setting.
  We further show that non-linear training is necessary: for this class of
functions, linear methods on any feature map (e.g., the NTK) are not capable of
learning efficiently. The key tools are a new "dimension-free" dynamics
approximation result that applies to functions defined on a latent space of
low-dimension, a proof of global convergence based on polynomial identity
testing, and an improvement of lower bounds against linear methods for
non-almost orthogonal functions.

14
The categories of this paper are: math-ph
35
The categories of this paper are: mtrl-th
9
The categories of this paper are: nlin
5
The categories of this paper are: gr-qc
10
The categories of this paper are: q-bio
33
The categories of this paper are: plasm-ph
29
The categories of this paper are: funct-an
36
The categories of this paper are: cmp-lg
2
The categories of this paper are: math
27
The categories of this paper are: comp-gas
23
The categories of this paper are: solv-int
20
The categories of this paper are: acc-phys
30
The categories of this paper are: q-alg
4
The categories of this paper are: cond-mat
24
The categories of this paper are: dg-ga
18
The categories of this paper are: econ