Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Towards Model Agnostic Federated Learning Using Knowledge Distillation   An often unquestioned assumption underlying most current federated learning
algorithms is that all the participants use identical model architectures. In
this work, we initiate a theoretical study of model agnostic communication
protocols which would allow data holders (agents) using different models to
collaborate with each other and perform federated learning. We focus on the
setting where the two agents are attempting to perform kernel regression using
different kernels (and hence have different models). Our study yields a
surprising result -- the most natural algorithm of using alternating knowledge
distillation (AKD) imposes overly strong regularization and may lead to severe
under-fitting. Our theory also shows an interesting connection between AKD and
the alternating projection algorithm for finding intersection of sets.
Leveraging this connection, we propose a new algorithms which improve upon AKD.
Our theoretical predictions also closely match real world experiments using
neural networks. Thus, our work proposes a rich yet tractable framework for
analyzing and developing new practical model agnostic federated learning
algorithms.

24
The categories of this paper are: dg-ga
12
The categories of this paper are: nucl-th
18
The categories of this paper are: econ
3
The categories of this paper are: physics
35
The categories of this paper are: mtrl-th
31
The categories of this paper are: ao-sci
27
The categories of this paper are: comp-gas
9
The categories of this paper are: nlin
7
The categories of this paper are: hep-th
23
The categories of this paper are: solv-int
36
The categories of this paper are: cmp-lg
33
The categories of this paper are: plasm-ph
15
The categories of this paper are: nucl-ex
19
The categories of this paper are: eess
21
The categories of this paper are: adap-org
32
The categories of this paper are: atom-ph