  Today, the largest Lustre file systems store billions of entries. On such
systems, classic tools based on namespace scanning become unusable. Operations
such as managing file lifetime, scheduling data copies, and generating overall
filesystem statistics become painful as they require collecting, sorting and
aggregating information for billions of records. Robinhood Policy Engine is an
open source software developed to address these challenges. It makes it
possible to schedule automatic actions on huge numbers of filesystem entries.
It also gives a synthetic understanding of file systems contents by providing
overall statistics about data ownership, age and size profiles. Even if it can
be used with any POSIX filesystem, Robinhood supports Lustre specific features
like OSTs, pools, HSM, ChangeLogs, and DNE. It implements specific support for
these features, and takes advantage of them to manage Lustre file systems
efficiently.
