Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Understanding Limitation of Two Symmetrized Orders by Worst-case
  Complexity   Update order is one of the major design choices of block decomposition
algorithms. There are at least two classes of deterministic update orders:
nonsymmetric (e.g. cyclic order) and symmetric (e.g. Gaussian back substitution
or symmetric Gauss-Seidel). Recently, Coordinate Descent (CD) with cyclic order
was shown to be $O(n^2)$ times slower than randomized versions in the
worst-case. A natural question arises: can the symmetrized orders achieve
faster convergence rates than the cyclic order, or even getting close to the
randomized versions? In this paper, we give a negative answer to this question.
We show that both Gaussian back substitution (GBS) and symmetric Gauss-Seidel
(sGS) suffer from the same slow convergence issue as the cyclic order in the
worst case. In particular, we prove that for unconstrained problems, both
GBS-CD and sGS-CD can be $O(n^2)$ times slower than R-CD. Despite unconstrained
problems, we also empirically study linearly constrained problems with
quadratic objective: we empirically demonstrate that the convergence speed of
GBS-ADMM and sGS-ADMM can be roughly $O(n^2)$ times slower than randomly
permuted ADMM.

30
The categories of this paper are: q-alg
24
The categories of this paper are: dg-ga
23
The categories of this paper are: solv-int
22
The categories of this paper are: chao-dyn
37
The categories of this paper are: supr-con
14
The categories of this paper are: math-ph
3
The categories of this paper are: physics
5
The categories of this paper are: gr-qc
25
The categories of this paper are: patt-sol
13
The categories of this paper are: quant-ph
31
The categories of this paper are: ao-sci
35
The categories of this paper are: mtrl-th
7
The categories of this paper are: hep-th
17
The categories of this paper are: q-fin
32
The categories of this paper are: atom-ph
27
The categories of this paper are: comp-gas