  The workloads running in the modern data centers of large scale Internet
service providers (such as Amazon, Baidu, Facebook, Google, and Microsoft)
support billions of users and span globally distributed infrastructure. Yet,
the devices used in modern data centers fail due to a variety of causes, from
faulty components to bugs to misconfiguration. Faulty devices make operating
large scale data centers challenging because the workloads running in modern
data centers consist of interdependent programs distributed across many
servers, so failures that are isolated to a single device can still have a
widespread effect on a workload. In this dissertation, we measure and model the
device failures in a large scale Internet service company, Facebook. We focus
on three device types that form the foundation of Internet service data center
infrastructure: DRAM for main memory, SSDs for persistent storage, and switches
and backbone links for network connectivity. For each of these device types, we
analyze long term device failure data broken down by important device
attributes and operating conditions, such as age, vendor, and workload. We also
build and release statistical models to examine the failure trends for the
devices we analyze. Our key conclusion in this dissertation is that we can gain
a deep understanding of why devices fail---and how to predict their
failure---using measurement and modeling. We hope that the analysis,
techniques, and models we present in this dissertation will enable the
community to better measure, understand, and prepare for the hardware
reliability challenges we face in the future.
