  AlphaFold predicts protein structures from the amino acid sequence at or near
experimental resolution, solving the 50-year-old protein folding challenge,
leading to progress by transforming large-scale genomics data into protein
structures. AlphaFold will also greatly change the scientific research model
from low-throughput to high-throughput manner. The AlphaFold framework is a
mixture of two types of workloads: MSA construction based on CPUs and model
inference on GPUs. The first CPU stage dominates the overall runtime, taking
hours for a single protein due to the large database sizes and I/O bottlenecks.
However, GPUs in this CPU stage remain idle, resulting in low GPU utilization
and restricting the capacity of large-scale structure predictions. Therefore,
we proposed ParaFold, an open-source parallel version of AlphaFold for high
throughput protein structure predictions. ParaFold separates the CPU and GPU
parts to enable large-scale structure predictions. ParaFold also effectively
reduces the CPU and GPU runtime with two optimizations without compromising the
quality of prediction results: using multi-threaded parallelism on CPUs and
using optimized JAX compilation on GPUs. We evaluated ParaFold with three
datasets of different size and protein lengths. We evaluated the accuracy and
efficiency of optimizations on CPUs and GPUs, and showed the large-scale
prediction capability by running ParaFold inferences of 19,704 small proteins
in five hours on one NVIDIA DGX-2. Using the JAX compile optimization, ParaFold
attained a 13.8X average speedup over AlphaFold. ParaFold offers a rapid and
effective approach for high-throughput structure predictions, leveraging the
predictive power by running on supercomputers, with shorter time, and at a
lower cost. The development of ParaFold will greatly speed up high-throughput
studies and render the protein "structure-omics" feasible.
