Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: The Impact of Automated Parameter Optimization on Defect Prediction
  Models   Defect prediction models---classifiers that identify defect-prone software
modules---have configurable parameters that control their characteristics
(e.g., the number of trees in a random forest). Recent studies show that these
classifiers underperform when default settings are used. In this paper, we
study the impact of automated parameter optimization on defect prediction
models. Through a case study of 18 datasets, we find that automated parameter
optimization: (1) improves AUC performance by up to 40 percentage points; (2)
yields classifiers that are at least as stable as those trained using default
settings; (3) substantially shifts the importance ranking of variables, with as
few as 28% of the top-ranked variables in optimized classifiers also being
top-ranked in non-optimized classifiers; (4) yields optimized settings for 17
of the 20 most sensitive parameters that transfer among datasets without a
statistically significant drop in performance; and (5) adds less than 30
minutes of additional computation to 12 of the 26 studied classification
techniques. While widely-used classification techniques like random forest and
support vector machines are not optimization-sensitive, traditionally
overlooked techniques like C5.0 and neural networks can actually outperform
widely-used techniques after optimization is applied. This highlights the
importance of exploring the parameter space when using parameter-sensitive
classification techniques.

20
The categories of this paper are: acc-phys
7
The categories of this paper are: hep-th
21
The categories of this paper are: adap-org
23
The categories of this paper are: solv-int
17
The categories of this paper are: q-fin
18
The categories of this paper are: econ
16
The categories of this paper are: stat
22
The categories of this paper are: chao-dyn
19
The categories of this paper are: eess
0
The categories of this paper are: hep-ph
27
The categories of this paper are: comp-gas
15
The categories of this paper are: nucl-ex
31
The categories of this paper are: ao-sci
9
The categories of this paper are: nlin
11
The categories of this paper are: hep-lat
28
The categories of this paper are: alg-geom