Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Replication studies considered harmful   CONTEXT: There is growing interest in establishing software engineering as an
evidence-based discipline. To that end, replication is often used to gain
confidence in empirical findings, as opposed to reproduction where the goal is
showing the correctness, or validity of the published results. OBJECTIVE: To
consider what is required for a replication study to confirm the original
experiment and apply this understanding in software engineering. METHOD:
Simulation is used to demonstrate why the prediction interval for confirmation
can be surprisingly wide. This analysis is applied to three recent
replications. RESULTS: It is shown that because the prediction intervals are
wide, almost all replications are confirmatory, so in that sense there is no
'replication crisis', however, the contributions to knowledge are negligible.
CONCLUSIONS: Replicating empirical software engineering experiments,
particularly if they are under-powered or under-reported, is a waste of
scientific resources. By contrast, meta-analysis is strongly advocated so that
all relevant experiments are combined to estimate the population effect.

15
The categories of this paper are: nucl-ex
26
The categories of this paper are: bayes-an
34
The categories of this paper are: chem-ph
17
The categories of this paper are: q-fin
33
The categories of this paper are: plasm-ph
7
The categories of this paper are: hep-th
20
The categories of this paper are: acc-phys
3
The categories of this paper are: physics
28
The categories of this paper are: alg-geom
5
The categories of this paper are: gr-qc
14
The categories of this paper are: math-ph
29
The categories of this paper are: funct-an
32
The categories of this paper are: atom-ph
35
The categories of this paper are: mtrl-th
25
The categories of this paper are: patt-sol
0
The categories of this paper are: hep-ph