  We present JAttack, a framework that enables template-based testing for
compilers. Using JAttack, a developer writes a template program that describes
a set of programs to be generated and given as test inputs to a compiler. Such
a framework enables developers to incorporate their domain knowledge on testing
compilers, giving a basic program structure that allows for exploring complex
programs that can trigger sophisticated compiler optimizations. A developer
writes a template program in the host language (Java) that contains holes to be
filled by JAttack. Each hole, written using a domain-specific language,
constructs a node within an extended abstract syntax tree (eAST). An eAST node
defines the search space for the hole, i.e., a set of expressions and values.
JAttack generates programs by executing templates and filling each hole by
randomly choosing expressions and values (available within the search space
defined by the hole). Additionally, we introduce several optimizations to
reduce JAttack's generation cost. While JAttack could be used to test various
compiler features, we demonstrate its capabilities in helping test just-in-time
(JIT) Java compilers, whose optimizations occur at runtime after a sufficient
number of executions. Using JAttack, we have found six critical bugs that were
confirmed by Oracle developers. Four of them were previously unknown, including
two unknown CVEs (Common Vulnerabilities and Exposures). JAttack shows the
power of combining developers' domain knowledge (via templates) with random
testing to detect bugs in JIT compilers.
