  Accompanying the successes of learning-based defensive software vulnerability
analyses is the lack of large and quality sets of labeled vulnerable program
samples, which impedes further advancement of those defenses. Existing
automated sample generation approaches have shown potentials yet still fall
short of practical expectations due to the high noise in the generated samples.
This paper proposes VGX, a new technique aimed for large-scale generation of
high-quality vulnerability datasets. Given a normal program, VGX identifies the
code contexts in which vulnerabilities can be injected, using a customized
Transformer featured with a new value-flowbased position encoding and
pre-trained against new objectives particularly for learning code structure and
context. Then, VGX materializes vulnerability-injection code editing in the
identified contexts using patterns of such edits obtained from both historical
fixes and human knowledge about real-world vulnerabilities. Compared to four
state-of-the-art (SOTA) baselines (pattern-, Transformer-, GNN-, and
pattern+Transformer-based), VGX achieved 99.09-890.06% higher F1 and
22.45%-328.47% higher label accuracy. For in-the-wild sample production, VGX
generated 150,392 vulnerable samples, from which we randomly chose 10% to
assess how much these samples help vulnerability detection, localization, and
repair. Our results show SOTA techniques for these three application tasks
achieved 19.15-330.80% higher F1, 12.86-19.31% higher top-10 accuracy, and
85.02-99.30% higher top-50 accuracy, respectively, by adding those samples to
their original training data. These samples also helped a SOTA vulnerability
detector discover 13 more real-world vulnerabilities (CVEs) in critical systems
(e.g., Linux kernel) that would be missed by the original model.
