  Dynamic parallelism on GPUs allows GPU threads to dynamically launch other
GPU threads. It is useful in applications with nested parallelism, particularly
where the amount of nested parallelism is irregular and cannot be predicted
beforehand. However, prior works have shown that dynamic parallelism may impose
a high performance penalty when a large number of small grids are launched. The
large number of launches results in high launch latency due to congestion, and
the small grid sizes result in hardware underutilization.
  To address this issue, we propose a compiler framework for optimizing the use
of dynamic parallelism in applications with nested parallelism. The framework
features three key optimizations: thresholding, coarsening, and aggregation.
Thresholding involves launching a grid dynamically only if the number of child
threads exceeds some threshold, and serializing the child threads in the parent
thread otherwise. Coarsening involves executing the work of multiple thread
blocks by a single coarsened block to amortize the common work across them.
Aggregation involves combining multiple child grids into a single aggregated
grid.
  Our evaluation shows that our compiler framework improves the performance of
applications with nested parallelism by a geometric mean of 43.0x over
applications that use dynamic parallelism, 8.7x over applications that do not
use dynamic parallelism, and 3.6x over applications that use dynamic
parallelism with aggregation alone as proposed in prior work.
