  Container technique is gaining increasing attention in recent years and has
become an alternative to traditional virtual machines. Some of the primary
motivations for the enterprise to adopt the container technology include its
convenience to encapsulate and deploy applications, lightweight operations, as
well as efficiency and flexibility in resources sharing. However, there still
lacks an in-depth and systematic comparison study on how big data applications,
such as Spark jobs, perform between a container environment and a virtual
machine environment. In this paper, by running various Spark applications with
different configurations, we evaluate the two environments from many
interesting aspects, such as how convenient the execution environment can be
set up, what are makespans of different workloads running in each setup, how
efficient the hardware resources, such as CPU and memory, are utilized, and how
well each environment can scale. The results show that compared with virtual
machines, containers provide a more easy-to-deploy and scalable environment for
big data workloads. The research work in this paper can help practitioners and
researchers to make more informed decisions on tuning their cloud environment
and configuring the big data applications, so as to achieve better performance
and higher resources utilization.
