Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Can We Benchmark Code Review Studies? A Systematic Mapping Study of
  Methodology, Dataset, and Metric   Code Review (CR) is the cornerstone for software quality assurance and a
crucial practice for software development. As CR research matures, it can be
difficult to keep track of the best practices and state-of-the-art in
methodology, dataset, and metric. This paper investigates the potential of
benchmarking by collecting methodology, dataset, and metric of CR studies. A
systematic mapping study was conducted. A total of 112 studies from 19,847
papers published in high-impact venues between the years 2011 and 2019 were
selected and analyzed. First, we find that empirical evaluation is the most
common methodology (65% of papers), with solution and experience being the
least common methodology. Second, we highlight 50% of papers that use the
quantitative method or mixed-method have the potential for replicability.
Third, we identify 457 metrics that are grouped into sixteen core metric sets,
applied to nine Software Engineering topics, showing different research topics
tend to use specific metric sets. We conclude that at this stage, we cannot
benchmark CR studies. Nevertheless, a common benchmark will facilitate new
researchers, including experts from other fields, to innovate new techniques
and build on top of already established methodologies. A full replication is
available at https://naist-se.github.io/code-review/.

14
The categories of this paper are: math-ph
29
The categories of this paper are: funct-an
13
The categories of this paper are: quant-ph
20
The categories of this paper are: acc-phys
2
The categories of this paper are: math
15
The categories of this paper are: nucl-ex
21
The categories of this paper are: adap-org
5
The categories of this paper are: gr-qc
8
The categories of this paper are: hep-ex
36
The categories of this paper are: cmp-lg
30
The categories of this paper are: q-alg
31
The categories of this paper are: ao-sci
0
The categories of this paper are: hep-ph
18
The categories of this paper are: econ
25
The categories of this paper are: patt-sol
19
The categories of this paper are: eess