Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Bilevel stochastic methods for optimization and machine learning:
  Bilevel stochastic descent and DARTS   Two-level stochastic optimization formulations have become instrumental in a
number of machine learning contexts such as neural architecture search,
continual learning, adversarial learning, and hyperparameter tuning. Practical
stochastic bilevel optimization problems become challenging in optimization or
learning scenarios where the number of variables is high or there are
constraints.
  The goal of this paper is twofold. First, we aim at promoting the use of
bilevel optimization in large-scale learning and we introduce a practical
bilevel stochastic gradient method (BSG-1) that requires neither lower level
second-order derivatives nor system solves (and dismisses any matrix-vector
products). Our BSG-1 method is close to first-order principles, which allows it
to achieve a performance better than those that are not, such as DARTS. Second,
we develop bilevel stochastic gradient descent for bilevel problems with lower
level constraints, and we introduce a convergence theory that covers the
unconstrained and constrained cases and abstracts as much as possible from the
specifics of the bilevel gradient calculation.

34
The categories of this paper are: chem-ph
32
The categories of this paper are: atom-ph
4
The categories of this paper are: cond-mat
26
The categories of this paper are: bayes-an
21
The categories of this paper are: adap-org
36
The categories of this paper are: cmp-lg
28
The categories of this paper are: alg-geom
33
The categories of this paper are: plasm-ph
7
The categories of this paper are: hep-th
0
The categories of this paper are: hep-ph
37
The categories of this paper are: supr-con
22
The categories of this paper are: chao-dyn
10
The categories of this paper are: q-bio
35
The categories of this paper are: mtrl-th
25
The categories of this paper are: patt-sol
30
The categories of this paper are: q-alg