  Infrastructure as code (IaC) scripts are used to automate the maintenance and
configuration of software development and deployment infrastructure. IaC
scripts can be complex in nature, containing hundreds of lines of code, leading
to defects that can be difficult to debug, and lead to wide-scale system
discrepancies such as service outages at scale. Use of IaC scripts is getting
increasingly popular, yet the nature of defects that occur in these scripts
have not been systematically categorized. A systematic categorization of
defects can inform practitioners about process improvement opportunities to
mitigate defects in IaC scripts. The goal of this paper is to help software
practitioners improve their development process of infrastructure as code (IaC)
scripts by categorizing the defect categories in IaC scripts based upon a
qualitative analysis of commit messages and issue report descriptions. We mine
open source version control systems collected from four organizations namely,
Mirantis, Mozilla, Openstack, and Wikimedia Commons to conduct our research
study. We use 1021, 3074, 7808, and 972 commits that map to 165, 580, 1383, and
296 IaC scripts, respectively, collected from Mirantis, Mozilla, Openstack, and
Wikimedia Commons. With 89 raters we apply the defect type attribute of the
orthogonal defect classification (ODC) methodology to categorize the defects.
We also review prior literature that have used ODC to categorize defects, and
compare the defect category distribution of IaC scripts with 26 non-IaC
software systems. Respectively, for Mirantis, Mozilla, Openstack, and Wikimedia
Commons, we observe (i) 49.3%, 36.5%, 57.6%, and 62.7% of the IaC defects to
contain syntax and configuration-related defects; (ii) syntax and
configuration-related defects are more prevalent amongst IaC scripts compared
to that of previously-studied non-IaC software.
