  Software debloating tools seek to improve the program security and
performance by removing unnecessary code, called bloat. While many techniques
have been proposed, several barriers to their adoption have emerged. Namely,
debloating tools are highly specialized, making it difficult for adopters to
find the right type of tool for their needs. This is further hindered by a lack
of established metrics and comparative evaluations between tools. To close this
gap, we surveyed of 10 years of debloating literature and several tools
currently under commercial development to systematize the debloating
ecosystem's knowledge. We then conducted a broad comparative evaluation of 10
debloating tools to determine their relative strengths and weaknesses. Our
evaluation, conducted on a diverse set of 20 benchmark programs, measures tools
across 16 performance, security, correctness, and usability metrics.
  Our evaluation surfaces several concerning findings that contradict the
prevailing narrative in debloating literature. First, debloating tools lack the
required maturity to be used on real-world software, evidenced by a slim 21%
overall success rate for creating passable debloated versions of medium- and
high-complexity benchmarks. Second, debloating tools struggle to produce sound
and robust programs. Using our novel differential fuzzing tool, DIFFER, we
discovered that only 13% of our debloating attempts produced a sound and robust
debloated program. Finally, our results indicate that debloating tools
typically do not improve the performance or security posture of debloated
programs by a significant degree. We believe that our contributions in this
paper will help potential adopters better understand the landscape of tools and
will motivate future research and development of more capable debloating tools.
To this end, we have made our benchmark set, data, and custom tools publicly
available.
