  There is an ongoing effort to develop tools that apply distributed
computational resources to tackle large problems or reduce the time to solve
them. In this context, the Alternating Direction Method of Multipliers (ADMM)
arises as a method that can exploit distributed resources like the dual ascent
method and has the robustness and improved convergence of the augmented
Lagrangian method. Traditional approaches to accelerate the ADMM using multiple
cores are problem-specific and often require multi-core programming. By
contrast, we propose a problem-independent scheme of accelerating the ADMM that
does not require the user to write any parallel code. We show that this scheme,
an interpretation of the ADMM as a message-passing algorithm on a factor-graph,
can automatically exploit fine-grained parallelism both in GPUs and
shared-memory multi-core computers and achieves significant speedup in such
diverse application domains as combinatorial optimization, machine learning,
and optimal control. Specifically, we obtain 10-18x speedup using a GPU, and
5-9x using multiple CPU cores, over a serial, optimized C-version of the ADMM,
which is similar to the typical speedup reported for existing GPU-accelerated
libraries, including cuFFT (19x), cuBLAS (17x), and cuRAND (8x).
