Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: The Common Intuition to Transfer Learning Can Win or Lose: Case Studies
  for Linear Regression   We study a fundamental transfer learning process from source to target linear
regression tasks, including overparameterized settings where there are more
learned parameters than data samples. The target task learning is addressed by
using its training data together with the parameters previously computed for
the source task. We define a transfer learning approach to the target task as a
linear regression optimization with a regularization on the distance between
the to-be-learned target parameters and the already-learned source parameters.
We analytically characterize the generalization performance of our transfer
learning approach and demonstrate its ability to resolve the peak in
generalization errors in double descent phenomena of the minimum $\ell_2$-norm
solution to linear regression. Moreover, we show that for sufficiently related
tasks, the optimally tuned transfer learning approach can outperform the
optimally tuned ridge regression method, even when the true parameter vector
conforms to an isotropic Gaussian prior distribution. Namely, we demonstrate
that transfer learning can beat the minimum mean square error (MMSE) solution
of the independent target task. Our results emphasize the ability of transfer
learning to extend the solution space to the target task and, by that, to have
an improved MMSE solution. We formulate the linear MMSE solution to our
transfer learning setting and point out its key differences from the common
design philosophy to transfer learning.

11
The categories of this paper are: hep-lat
17
The categories of this paper are: q-fin
15
The categories of this paper are: nucl-ex
27
The categories of this paper are: comp-gas
24
The categories of this paper are: dg-ga
25
The categories of this paper are: patt-sol
7
The categories of this paper are: hep-th
21
The categories of this paper are: adap-org
34
The categories of this paper are: chem-ph
36
The categories of this paper are: cmp-lg
9
The categories of this paper are: nlin
23
The categories of this paper are: solv-int
0
The categories of this paper are: hep-ph
30
The categories of this paper are: q-alg
10
The categories of this paper are: q-bio
37
The categories of this paper are: supr-con