  Due to the promising future of Automated Program Repair (APR), researchers
have proposed various APR techniques, including heuristic-based,
template-based, and constraint-based techniques. Among such classic APR
techniques, template-based techniques have been widely recognized as state of
the art. However, such template-based techniques require predefined templates
to perform repair, and their effectiveness is thus limited. To this end,
researchers leveraged the recent advances in Deep Learning to further improve
APR. Such learning-based techniques view APR as a Neural Machine Translation
problem, using the buggy/fixed code snippets as the source/target languages for
translation. In this way, such techniques heavily rely on large numbers of
high-quality bug-fixing commits, which can be extremely costly and challenging
to construct. Furthermore, the edit variety of these learning-based techniques
are limited to the available bug-fixes within their training datasets.
Therefore, in this paper, we aim to revisit the learning-based APR problem, and
propose AlphaRepair, to leverage zero-shot learning directly using large
pre-trained code models for APR. Our main insight is instead of modeling what a
repair edit should look like, we can directly predict what the correct code is
based on the context information. We have implemented AlphaRepair as a
practical multilingual APR tool based on the recent CodeBERT model. Our results
on the widely used Defects4J benchmark show that AlphaRepair can substantially
outperform state-of-the-art APR tools. We also studied the impact of different
design choices and show that AlphaRepair performs even better on a newer
version of Defects4J (2.0) with 3.3X more fixes than best performing baseline,
indicating that AlphaRepair can potentially avoid the dataset-overfitting issue
of existing learning-based techniques.
