Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Communication-Efficient Decentralized Learning with Sparsification and
  Adaptive Peer Selection   Distributed learning techniques such as federated learning have enabled
multiple workers to train machine learning models together to reduce the
overall training time. However, current distributed training algorithms
(centralized or decentralized) suffer from the communication bottleneck on
multiple low-bandwidth workers (also on the server under the centralized
architecture). Although decentralized algorithms generally have lower
communication complexity than the centralized counterpart, they still suffer
from the communication bottleneck for workers with low network bandwidth. To
deal with the communication problem while being able to preserve the
convergence performance, we introduce a novel decentralized training algorithm
with the following key features: 1) It does not require a parameter server to
maintain the model during training, which avoids the communication pressure on
any single peer. 2) Each worker only needs to communicate with a single peer at
each communication round with a highly compressed model, which can
significantly reduce the communication traffic on the worker. We theoretically
prove that our sparsification algorithm still preserves convergence properties.
3) Each worker dynamically selects its peer at different communication rounds
to better utilize the bandwidth resources. We conduct experiments with
convolutional neural networks on 32 workers to verify the effectiveness of our
proposed algorithm compared to seven existing methods. Experimental results
show that our algorithm significantly reduces the communication traffic and
generally select relatively high bandwidth peers.

4
The categories of this paper are: cond-mat
7
The categories of this paper are: hep-th
9
The categories of this paper are: nlin
20
The categories of this paper are: acc-phys
26
The categories of this paper are: bayes-an
29
The categories of this paper are: funct-an
12
The categories of this paper are: nucl-th
2
The categories of this paper are: math
31
The categories of this paper are: ao-sci
18
The categories of this paper are: econ
33
The categories of this paper are: plasm-ph
19
The categories of this paper are: eess
0
The categories of this paper are: hep-ph
28
The categories of this paper are: alg-geom
23
The categories of this paper are: solv-int
25
The categories of this paper are: patt-sol