  Growing heterogeneity and configurability in HPC architectures has made
auto-tuning applications and runtime parameters on these systems very complex.
Users are presented with a multitude of options to configure parameters. In
addition to application specific solutions, a common approach is to use general
purpose search strategies, which often might not identify the best
configurations or their time to convergence is a significant barrier. There is,
thus, a need for a general purpose and efficient tuning approach that can be
easily scaled and adapted to various tuning tasks. We propose a technique for
tuning parallel code regions that is general enough to be adapted to multiple
tasks. In this paper, we analyze IR-based programming models to make
task-specific performance optimizations. To this end, we propose the Multimodal
Graph Neural Network and Autoencoder (MGA) tuner, a multimodal deep learning
based approach that adapts Heterogeneous Graph Neural Networks and Denoizing
Autoencoders for modeling IR-based code representations that serve as separate
modalities. This approach is used as part of our pipeline to model a syntax,
semantics, and structure-aware IR-based code representation for tuning parallel
code regions/kernels. We extensively experiment on OpenMP and OpenCL code
regions/kernels obtained from PolyBench, Rodinia, STREAM, DataRaceBench, AMD
SDK, NPB, NVIDIA SDK, Parboil, SHOC, and LULESH benchmarks. We apply our
multimodal learning techniques to the tasks of i) optimizing the number of
threads, scheduling policy and chunk size in OpenMP loops and, ii) identifying
the best device for heterogeneous device mapping of OpenCL kernels. Our
experiments show that this multimodal learning based approach outperforms the
state-of-the-art in all experiments.
