text: In this paper, we address the problem of how a network of agents can collaboratively fit a linear model when each agent only ever has an arbitrary summand of the regression data. This problem generalizes previously studied data-matrix-splitting scenarios, allowing for some agents to have more measurements of some features than of others and even have measurements that other agents have. We present a variable-centric framework for distributed optimization in a network, and use this framework to develop a proximal algorithm, based on the Douglas-Rachford method, that solves the problem.
  @cite_5: This book provides a largely self-contained account of the main results of convex analysis and optimization in Hilbert space. A concise exposition of related constructive fixed point theory is presented, that allows for a wide range of algorithms to construct solutions to problems in optimization, equilibrium theory, monotone inclusions, variational inequalities, best approximation theory, and convex feasibility. The book is accessible to a broad audience, and reaches out in particular to applied scientists and engineers, to whom these tools have become indispensable.
  @cite_3: Many problems of recent interest in statistics and machine learning can be posed in the framework of convex optimization. Due to the explosion in size and complexity of modern datasets, it is increasingly important to be able to solve problems with a very large number of features or training examples. As a result, both the decentralized collection or storage of these datasets as well as accompanying distributed solution methods are either necessary or at least highly desirable. In this review, we argue that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas. The method was developed in the 1970s, with roots in the 1950s, and is equivalent or closely related to many other algorithms, such as dual decomposition, the method of multipliers, Douglas–Rachford splitting, Spingarn's method of partial inverses, Dykstra's alternating projections, Bregman iterative algorithms for l1 problems, proximal methods, and others. After briefly surveying the theory and history of the algorithm, we discuss applications to a wide variety of statistical and machine learning problems of recent interest, including the lasso, sparse logistic regression, basis pursuit, covariance selection, support vector machines, and many others. We also discuss general distributed optimization, extensions to the nonconvex setting, and efficient implementation, including some details on distributed MPI and Hadoop MapReduce implementations.
summary: In solving @math , the algorithm we provide is based on the Douglas-Rachford proximal splitting method @cite_5 . It falls in a growing body of recent work on applying splitting methods, like the alternating directions method of multipliers (ADMM), to obtain distributed algorithms; such work is perhaps most inspired by @cite_3 .