Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: A Theoretical Analysis of Catastrophic Forgetting through the NTK
  Overlap Matrix   Continual learning (CL) is a setting in which an agent has to learn from an
incoming stream of data during its entire lifetime. Although major advances
have been made in the field, one recurring problem which remains unsolved is
that of Catastrophic Forgetting (CF). While the issue has been extensively
studied empirically, little attention has been paid from a theoretical angle.
In this paper, we show that the impact of CF increases as two tasks
increasingly align. We introduce a measure of task similarity called the NTK
overlap matrix which is at the core of CF. We analyze common projected gradient
algorithms and demonstrate how they mitigate forgetting. Then, we propose a
variant of Orthogonal Gradient Descent (OGD) which leverages structure of the
data through Principal Component Analysis (PCA). Experiments support our
theoretical findings and show how our method can help reduce CF on classical CL
datasets.

23
The categories of this paper are: solv-int
19
The categories of this paper are: eess
6
The categories of this paper are: astro-ph
36
The categories of this paper are: cmp-lg
37
The categories of this paper are: supr-con
2
The categories of this paper are: math
17
The categories of this paper are: q-fin
29
The categories of this paper are: funct-an
14
The categories of this paper are: math-ph
4
The categories of this paper are: cond-mat
32
The categories of this paper are: atom-ph
35
The categories of this paper are: mtrl-th
3
The categories of this paper are: physics
28
The categories of this paper are: alg-geom
25
The categories of this paper are: patt-sol
0
The categories of this paper are: hep-ph