  Virtual machines and virtualized hardware have been around for over half a
century. The commoditization of the x86 platform and its rapidly growing
hardware capabilities have led to recent exponential growth in the use of
virtualization both in the enterprise and high performance computing (HPC). The
startup time of a virtualized environment is a key performance metric for high
performance computing in which the runtime of any individual task is typically
much shorter than the lifetime of a virtualized service in an enterprise
context. In this paper, a methodology for accurately measuring the startup
performance on an HPC system is described. The startup performance overhead of
three of the most mature, widely deployed cloud management frameworks
(OpenStack, OpenNebula, and Eucalyptus) is measured to determine their
suitability for workloads typically seen in an HPC environment. A 10x
performance difference is observed between the fastest (Eucalyptus) and the
slowest (OpenNebula) framework. This time difference is primarily due to delays
in waiting on networking in the cloud-init portion of the startup. The
methodology and measurements presented should facilitate the optimization of
startup across a variety of virtualization environments.
