Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Basis Matters: Better Communication-Efficient Second Order Methods for
  Federated Learning   Recent advances in distributed optimization have shown that Newton-type
methods with proper communication compression mechanisms can guarantee fast
local rates and low communication cost compared to first order methods. We
discover that the communication cost of these methods can be further reduced,
sometimes dramatically so, with a surprisingly simple trick: {\em Basis Learn
(BL)}. The idea is to transform the usual representation of the local Hessians
via a change of basis in the space of matrices and apply compression tools to
the new representation. To demonstrate the potential of using custom bases, we
design a new Newton-type method (BL1), which reduces communication cost via
both {\em BL} technique and bidirectional compression mechanism. Furthermore,
we present two alternative extensions (BL2 and BL3) to partial participation to
accommodate federated learning applications. We prove local linear and
superlinear rates independent of the condition number. Finally, we support our
claims with numerical experiments by comparing several first and
second~order~methods.

0
The categories of this paper are: hep-ph
8
The categories of this paper are: hep-ex
21
The categories of this paper are: adap-org
27
The categories of this paper are: comp-gas
14
The categories of this paper are: math-ph
37
The categories of this paper are: supr-con
32
The categories of this paper are: atom-ph
31
The categories of this paper are: ao-sci
13
The categories of this paper are: quant-ph
34
The categories of this paper are: chem-ph
22
The categories of this paper are: chao-dyn
10
The categories of this paper are: q-bio
35
The categories of this paper are: mtrl-th
33
The categories of this paper are: plasm-ph
20
The categories of this paper are: acc-phys
9
The categories of this paper are: nlin