Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Understanding Loss Landscapes of Neural Network Models in Solving
  Partial Differential Equations   Solving partial differential equations (PDEs) by parametrizing its solution
by neural networks (NNs) has been popular in the past a few years. However,
different types of loss functions can be proposed for the same PDE. For the
Poisson equation, the loss function can be based on the weak formulation of
energy variation or the least squares method, which leads to the deep Ritz
model and deep Galerkin model, respectively. But loss landscapes from these
different models give arise to different practical performance of training the
NN parameters. To investigate and understand such practical differences, we
propose to compare the loss landscapes of these models, which are both high
dimensional and highly non-convex. In such settings, the roughness is more
important than the traditional eigenvalue analysis to describe the
non-convexity. We contribute to the landscape comparisons by proposing a
roughness index to scientifically and quantitatively describe the heuristic
concept of "roughness" of landscape around minimizers. This index is based on
random projections and the variance of (normalized) total variation for one
dimensional projected functions, and it is efficient to compute. A large
roughness index hints an oscillatory landscape profile as a severe challenge
for the first order optimization method. We apply this index to the two models
for the Poisson equation and our empirical results reveal a consistent general
observation that the landscapes from the deep Galerkin method around its local
minimizers are less rough than the deep Ritz method, which supports the
observed gain in accuracy of the deep Galerkin method.

37
The categories of this paper are: supr-con
27
The categories of this paper are: comp-gas
21
The categories of this paper are: adap-org
24
The categories of this paper are: dg-ga
23
The categories of this paper are: solv-int
8
The categories of this paper are: hep-ex
9
The categories of this paper are: nlin
25
The categories of this paper are: patt-sol
20
The categories of this paper are: acc-phys
6
The categories of this paper are: astro-ph
10
The categories of this paper are: q-bio
11
The categories of this paper are: hep-lat
16
The categories of this paper are: stat
29
The categories of this paper are: funct-an
35
The categories of this paper are: mtrl-th
4
The categories of this paper are: cond-mat