Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Collaborative Deep Learning in Fixed Topology Networks   There is significant recent interest to parallelize deep learning algorithms
in order to handle the enormous growth in data and model sizes. While most
advances focus on model parallelization and engaging multiple computing agents
via using a central parameter server, aspect of data parallelization along with
decentralized computation has not been explored sufficiently. In this context,
this paper presents a new consensus-based distributed SGD (CDSGD) (and its
momentum variant, CDMSGD) algorithm for collaborative deep learning over fixed
topology networks that enables data parallelization as well as decentralized
computation. Such a framework can be extremely useful for learning agents with
access to only local/private data in a communication constrained environment.
We analyze the convergence properties of the proposed algorithm with strongly
convex and nonconvex objective functions with fixed and diminishing step sizes
using concepts of Lyapunov function construction. We demonstrate the efficacy
of our algorithms in comparison with the baseline centralized SGD and the
recently proposed federated averaging algorithm (that also enables data
parallelism) based on benchmark datasets such as MNIST, CIFAR-10 and CIFAR-100.

13
The categories of this paper are: quant-ph
31
The categories of this paper are: ao-sci
25
The categories of this paper are: patt-sol
23
The categories of this paper are: solv-int
4
The categories of this paper are: cond-mat
8
The categories of this paper are: hep-ex
35
The categories of this paper are: mtrl-th
14
The categories of this paper are: math-ph
0
The categories of this paper are: hep-ph
28
The categories of this paper are: alg-geom
11
The categories of this paper are: hep-lat
17
The categories of this paper are: q-fin
22
The categories of this paper are: chao-dyn
30
The categories of this paper are: q-alg
36
The categories of this paper are: cmp-lg
2
The categories of this paper are: math