Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Towards Gradient-based Bilevel Optimization with Non-convex Followers
  and Beyond   In recent years, Bi-Level Optimization (BLO) techniques have received
extensive attentions from both learning and vision communities. A variety of
BLO models in complex and practical tasks are of non-convex follower structure
in nature (a.k.a., without Lower-Level Convexity, LLC for short). However, this
challenging class of BLOs is lack of developments on both efficient solution
strategies and solid theoretical guarantees. In this work, we propose a new
algorithmic framework, named Initialization Auxiliary and Pessimistic
Trajectory Truncated Gradient Method (IAPTT-GM), to partially address the above
issues. In particular, by introducing an auxiliary as initialization to guide
the optimization dynamics and designing a pessimistic trajectory truncation
operation, we construct a reliable approximate version of the original BLO in
the absence of LLC hypothesis. Our theoretical investigations establish the
convergence of solutions returned by IAPTT-GM towards those of the original BLO
without LLC. As an additional bonus, we also theoretically justify the quality
of our IAPTT-GM embedded with Nesterov's accelerated dynamics under LLC. The
experimental results confirm both the convergence of our algorithm without LLC,
and the theoretical findings under LLC.

12
The categories of this paper are: nucl-th
6
The categories of this paper are: astro-ph
26
The categories of this paper are: bayes-an
16
The categories of this paper are: stat
33
The categories of this paper are: plasm-ph
13
The categories of this paper are: quant-ph
35
The categories of this paper are: mtrl-th
14
The categories of this paper are: math-ph
17
The categories of this paper are: q-fin
11
The categories of this paper are: hep-lat
21
The categories of this paper are: adap-org
19
The categories of this paper are: eess
15
The categories of this paper are: nucl-ex
25
The categories of this paper are: patt-sol
18
The categories of this paper are: econ
4
The categories of this paper are: cond-mat