text: Low-rank approximation of a matrix by means of random sampling has been consistently efficient in its empirical studies by many scientists who applied it with various sparse and structured multipliers, but adequate formal support for this empirical phenomenon has been missing so far. Our novel insight into the subject leads to such an elusive formal support and promises significant acceleration of the known algorithms for some fundamental problems of matrix computations and data mining and analysis. Our formal results and our numerical tests are in good accordance with each other. We also outline extensions of low-rank approximation algorithms and of our progress to the Least Squares Regression, the Fast Multipole Method, and the Conjugate Gradient algorithms.
  @cite_36: Low-rank matrix approximations, such as the truncated singular value decomposition and the rank-revealing QR decomposition, play a central role in data analysis and scientific computing. This work surveys and extends recent research which demonstrates that randomization offers a powerful tool for performing low-rank matrix approximation. These techniques exploit modern computational architectures more fully than classical methods and open the possibility of dealing with truly massive data sets. This paper presents a modular framework for constructing randomized algorithms that compute partial matrix decompositions. These methods use random sampling to identify a subspace that captures most of the action of a matrix. The input matrix is then compressed—either explicitly or implicitly—to this subspace, and the reduced matrix is manipulated deterministically to obtain the desired low-rank factorization. In many cases, this approach beats its classical competitors in terms of accuracy, robustness, and or speed. These claims are supported by extensive numerical experiments and a detailed error analysis. The specific benefits of randomized techniques depend on the computational environment. Consider the model problem of finding the @math dominant components of the singular value decomposition of an @math matrix. (i) For a dense input matrix, randomized algorithms require @math floating-point operations (flops) in contrast to @math for classical algorithms. (ii) For a sparse input matrix, the flop count matches classical Krylov subspace methods, but the randomized approach is more robust and can easily be reorganized to exploit multiprocessor architectures. (iii) For a matrix that is too large to fit in fast memory, the randomized techniques require only a constant number of passes over the data, as opposed to @math passes for classical algorithms. In fact, it is sometimes possible to perform matrix approximation with a single pass over the data.
summary: Part (ii) of our Theorem is implied by [Theorem 10.8] HMT11 , but our specific supporting estimates are more compact, cover the case of any @math (whereas @cite_36 assumes that @math ), and we deduce them by using a shorter proof (see Remark ). Our approach, our results listed in Section , some of our techniques, e.g., expan -sion compres -sion in Section , and even the concept of factor-Gaussian matrices are new, and so are our families of multipliers and policies of their generation, combination, and application in Sections and as well. Moreover, our progress can be extended to a variety of important matrix computations. In Section (Conclusions) we outline such novel extensions to the highly popular and much studied computations for Least Squares Regression, the Fast Multipole Method and the Conjugate Gradient Algorithms. Hereafter we use the acronyms LSR" for Least Squares Regression", FMM" for Fast Multipole Method", and CG " for Conjugate Gradient". The extensions provide new insights and new opportunities and should motivate further effort and further progress.