  Compiler bugs pose a significant threat to safety-critical applications, and
promptly and effectively isolating these bugs is crucial for assuring the
quality of compilers. However, the limited availability of debugging
information on reported bugs complicates the compiler bug isolation task.
Existing compiler bug isolation approaches typically convert the problem into a
test program mutation problem, but they are still limited by ineffective
mutation strategies or high human effort requirements. Drawing inspiration from
the recent progress of pre-trained Large Language Models (LLMs), such as
ChatGPT, in code generation, we propose a new approach named LLM4CBI to tame
LLMs to generate effective test programs for compiler bug isolation. However,
using LLMs directly for test program mutation may not yield the desired results
due to the challenges associated with formulating precise prompts and selecting
specialized prompts. To overcome the challenges, three new components are
designed in LLM4CBI. (1) LLM4CBI utilizes a program complexity-guided prompt
production component, which leverages data and control flow analysis to
identify the most valuable variables and locations in programs for mutation.
(2) LLM4CBI employs a memorized prompt selection component, which adopts
reinforcement learning to select specialized prompts for mutating test programs
continuously. (3) A test program validation component is proposed to select
specialized feedback prompts to avoid repeating the same mistakes during the
mutation process. Compared with the state-of-the-art approaches (DiWi and
RecBi), our evaluation demonstrates the advantages of LLM4CBI: It isolates more
bugs, ranging from 13.6% to 90.9% in various settings, than the other
approaches. Additionally, we demonstrate that LLM4CBI is extensible, allowing
for easy integration with other LLMs.
