Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Solon: Communication-efficient Byzantine-resilient Distributed Training
  via Redundant Gradients   There has been a growing need to provide Byzantine-resilience in distributed
model training. Existing robust distributed learning algorithms focus on
developing sophisticated robust aggregators at the parameter servers, but pay
less attention to balancing the communication cost and robustness. In this
paper, we propose Solon, an algorithmic framework that exploits gradient
redundancy to provide communication efficiency and Byzantine robustness
simultaneously. Our theoretical analysis shows a fundamental trade-off among
computational load, communication cost, and Byzantine robustness. We also
develop a concrete algorithm to achieve the optimal trade-off, borrowing ideas
from coding theory and sparse recovery. Empirical experiments on various
datasets demonstrate that Solon provides significant speedups over existing
methods to achieve the same accuracy, over 10 times faster than Bulyan and 80%
faster than Draco. We also show that carefully designed Byzantine attacks break
Signum and Bulyan, but do not affect the successful convergence of Solon.

14
The categories of this paper are: math-ph
7
The categories of this paper are: hep-th
34
The categories of this paper are: chem-ph
35
The categories of this paper are: mtrl-th
27
The categories of this paper are: comp-gas
5
The categories of this paper are: gr-qc
28
The categories of this paper are: alg-geom
37
The categories of this paper are: supr-con
25
The categories of this paper are: patt-sol
4
The categories of this paper are: cond-mat
3
The categories of this paper are: physics
12
The categories of this paper are: nucl-th
18
The categories of this paper are: econ
8
The categories of this paper are: hep-ex
15
The categories of this paper are: nucl-ex
16
The categories of this paper are: stat