  While modern parallel computing systems offer high performance, utilizing
these powerful computing resources to the highest possible extent demands
advanced knowledge of various hardware architectures and parallel programming
models. Furthermore, optimized software execution on parallel computing systems
demands consideration of many parameters at compile-time and run-time.
Determining the optimal set of parameters in a given execution context is a
complex task, and therefore to address this issue researchers have proposed
different approaches that use heuristic search or machine learning. In this
paper, we undertake a systematic literature review to aggregate, analyze and
classify the existing software optimization methods for parallel computing
systems. We review approaches that use machine learning or meta-heuristics for
software optimization at compile-time and run-time. Additionally, we discuss
challenges and future research directions. The results of this study may help
to better understand the state-of-the-art techniques that use machine learning
and meta-heuristics to deal with the complexity of software optimization for
parallel computing systems. Furthermore, it may aid in understanding the
limitations of existing approaches and identification of areas for improvement.
