  Context: Test-driven development (TDD) is an agile software development
approach that has been widely claimed to improve software quality. However, the
extent to which TDD improves quality appears to be largely dependent upon the
characteristics of the study in which it is evaluated (e.g., the research
method, participant type, programming environment, etc.). The particularities
of each study make the aggregation of results untenable. Objectives: The goal
of this paper is to: increase the accuracy and generalizability of the results
achieved in isolated experiments on TDD, provide joint conclusions on the
performance of TDD across different industrial and academic settings, and
assess the extent to which the characteristics of the experiments affect the
quality-related performance of TDD. Method: We conduct a family of 12
experiments on TDD in academia and industry. We aggregate their results by
means of meta-analysis. We perform exploratory analyses to identify variables
impacting the quality-related performance of TDD. Results: TDD novices achieve
a slightly higher code quality with iterative test-last development (i.e., ITL,
the reverse approach of TDD) than with TDD. The task being developed largely
determines quality. The programming environment, the order in which TDD and ITL
are applied, or the learning effects from one development approach to another
do not appear to affect quality. The quality-related performance of
professionals using TDD drops more than for students. We hypothesize that this
may be due to their being more resistant to change and potentially less
motivated than students. Conclusion: Previous studies seem to provide
conflicting results on TDD performance (i.e., positive vs. negative,
respectively). We hypothesize that these conflicting results may be due to
different study durations, experiment participants being unfamiliar with the
TDD process...
