Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Continual learning: a feature extraction formalization, an efficient
  algorithm, and fundamental obstructions   Continual learning is an emerging paradigm in machine learning, wherein a
model is exposed in an online fashion to data from multiple different
distributions (i.e. environments), and is expected to adapt to the distribution
change. Precisely, the goal is to perform well in the new environment, while
simultaneously retaining the performance on the previous environments (i.e.
avoid "catastrophic forgetting") -- without increasing the size of the model.
  While this setup has enjoyed a lot of attention in the applied community,
there hasn't be theoretical work that even formalizes the desired guarantees.
In this paper, we propose a framework for continual learning through the
framework of feature extraction -- namely, one in which features, as well as a
classifier, are being trained with each environment. When the features are
linear, we design an efficient gradient-based algorithm $\mathsf{DPGD}$, that
is guaranteed to perform well on the current environment, as well as avoid
catastrophic forgetting. In the general case, when the features are non-linear,
we show such an algorithm cannot exist, whether efficient or not.

37
The categories of this paper are: supr-con
15
The categories of this paper are: nucl-ex
17
The categories of this paper are: q-fin
23
The categories of this paper are: solv-int
33
The categories of this paper are: plasm-ph
27
The categories of this paper are: comp-gas
3
The categories of this paper are: physics
32
The categories of this paper are: atom-ph
35
The categories of this paper are: mtrl-th
28
The categories of this paper are: alg-geom
24
The categories of this paper are: dg-ga
8
The categories of this paper are: hep-ex
7
The categories of this paper are: hep-th
25
The categories of this paper are: patt-sol
36
The categories of this paper are: cmp-lg
2
The categories of this paper are: math