Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Translating Numerical Concepts for PDEs into Neural Architectures   We investigate what can be learned from translating numerical algorithms into
neural networks. On the numerical side, we consider explicit, accelerated
explicit, and implicit schemes for a general higher order nonlinear diffusion
equation in 1D, as well as linear multigrid methods. On the neural network
side, we identify corresponding concepts in terms of residual networks
(ResNets), recurrent networks, and U-nets. These connections guarantee
Euclidean stability of specific ResNets with a transposed convolution layer
structure in each block. We present three numerical justifications for skip
connections: as time discretisations in explicit schemes, as extrapolation
mechanisms for accelerating those methods, and as recurrent connections in
fixed point solvers for implicit schemes. Last but not least, we also motivate
uncommon design choices such as nonmonotone activation functions. Our findings
give a numerical perspective on the success of modern neural network
architectures, and they provide design criteria for stable networks.

20
The categories of this paper are: acc-phys
9
The categories of this paper are: nlin
0
The categories of this paper are: hep-ph
11
The categories of this paper are: hep-lat
21
The categories of this paper are: adap-org
14
The categories of this paper are: math-ph
33
The categories of this paper are: plasm-ph
3
The categories of this paper are: physics
36
The categories of this paper are: cmp-lg
4
The categories of this paper are: cond-mat
10
The categories of this paper are: q-bio
17
The categories of this paper are: q-fin
5
The categories of this paper are: gr-qc
19
The categories of this paper are: eess
15
The categories of this paper are: nucl-ex
30
The categories of this paper are: q-alg