  We present a study of crash-consistency bugs in persistent-memory (PM) file
systems and analyze their implications for file-system design and testing crash
consistency. We develop FlyTrap, a framework to test PM file systems for
crash-consistency bugs. FlyTrap discovered 18 new bugs across four PM file
systems; the bugs have been confirmed by developers and many have been already
fixed. The discovered bugs have serious consequences such as breaking the
atomicity of rename or making the file system unmountable. We present a
detailed study of the bugs we found and discuss some important lessons from
these observations. For instance, one of our findings is that many of the bugs
are due to logic errors, rather than errors in using flushes or fences; this
has important applications for future work on testing PM file systems. Another
key finding is that many bugs arise from attempts to improve efficiency by
performing metadata updates in-place and that recovery code that deals with
rebuilding in-DRAM state is a significant source of bugs. These observations
have important implications for designing and testing PM file systems. Our code
is available at https://github.com/utsaslab/flytrap .
