  Big data applications and analytics are employed in many sectors for a
variety of goals: improving customers satisfaction, predicting market behavior
or improving processes in public health. These applications consist of complex
software stacks that are often run on cloud systems. Predicting execution times
is important for estimating the cost of cloud services and for effectively
managing the underlying resources at runtime. Machine Learning (ML), providing
black box solutions to model the relationship between application performance
and system configuration without requiring in-detail knowledge of the system,
has become a popular way of predicting the performance of big data
applications. We investigate the cost-benefits of using supervised ML models
for predicting the performance of applications on Spark, one of today's most
widely used frameworks for big data analysis. We compare our approach with
\textit{Ernest} (an ML-based technique proposed in the literature by the Spark
inventors) on a range of scenarios, application workloads, and cloud system
configurations. Our experiments show that Ernest can accurately estimate the
performance of very regular applications, but it fails when applications
exhibit more irregular patterns and/or when extrapolating on bigger data set
sizes. Results show that our models match or exceed Ernest's performance,
sometimes enabling us to reduce the prediction error from 126-187% to only
5-19%.
