  Background: Despite the widespread use of automated security defect detection
tools, software projects still contain many security defects that could result
in serious damage. Such tools are largely context-insensitive and may not cover
all possible scenarios in testing potential issues, which makes them
susceptible to missing complex security defects. Hence, thorough detection
entails a synergistic cooperation between these tools and human-intensive
detection techniques, including code review. Code review is widely recognized
as a crucial and effective practice for identifying security defects. Aim: This
work aims to empirically investigate security defect detection through code
review. Method: To this end, we conducted an empirical study by analyzing code
review comments derived from four projects in the OpenStack and Qt communities.
Through manually checking 20,995 review comments obtained by keyword-based
search, we identified 614 comments as security-related. Results: Our results
show that (1) security defects are not prevalently discussed in code review,
(2) more than half of the reviewers provided explicit fixing
strategies/solutions to help developers fix security defects, (3) developers
tend to follow reviewers' suggestions and action the changes, (4) Not worth
fixing the defect now and Disagreement between the developer and the reviewer
are the main causes for not resolving security defects. Conclusions: Our
research results demonstrate that (1) software security practices should
combine manual code review with automated detection tools, achieving a more
comprehensive coverage to identifying and addressing security defects, and (2)
promoting appropriate standardization of practitioners' behaviors during code
review remains necessary for enhancing software security.
