  The demand for high-performance computing (HPC) is ever-increasing for
everyday statistical computing purposes. The downside is that we need to write
specialized code for each HPC environment. CPU-level parallelization needs to
be explicitly coded for effective use of multiple nodes in cluster
supercomputing environments. Acceleration via graphics processing units (GPUs)
requires to write kernel code. The Julia software package DistStat.jl
implements a data structure for distributed arrays that work on both multi-node
CPU clusters and multi-GPU environments transparently. This package paves a way
to developing high-performance statistical software in various HPC environments
simultaneously. As a demonstration of the transparency and scalability of the
package, we provide applications to large-scale nonnegative matrix
factorization, multidimensional scaling, and $\ell_1$-regularized Cox
proportional hazards model on an 8-GPU workstation and a 720-CPU-core virtual
cluster in Amazon Web Services (AWS) cloud. As a case in point, we analyze the
on-set of type-2 diabetes from the UK Biobank with 400,000 subjects and 500,000
single nucleotide polymorphisms using the $\ell_1$-regularized Cox proportional
hazards model. Fitting a half-million-variate regression model took less than
50 minutes on AWS.
