RobinHood FS Monitor is a tool for monitoring and purging large filesystems. It is available under theÂ CeCILL-Clicense, which is a French transposition of the GNU LGPLÂ and is fully LGPL-compatible.
Its main purpose is to collect statistics about file system content and trigger purge when disk usage exceeds a given threshold.
RobinHood is designed to perform all its tasks in parallel, so it is particularly adapted for managing large filesystems with millions of entries and petabytes of data.
- Disk space fair-share
- Policy Engine for temporary filesystems and Lustre-HSM:
- Purge policies
- Migration policies
- Deferred removal policy
- Directories cleanup policy
- Attribute-based policies, possibly using sophisticated boolean expressions
- Accounting and monitoring
- Custom alerts on filesystem entries
- Aware of Lustre OSTs and pools
- Since version 2.0, Robinhood uses a database engine for managing its list of entries, which offers a lot of benefits:
- It can manage larger filesystems, because the size of its list is no more limited by the memory of the machine.
- Purges can be performed immedialty, even after a system crash, because the list is persistent.
- Robinhood v2 can be split in several daemons running on several nodes, for more scalability: there is no direct communication between several robinhood instances, they are only clients of the same remote database.
- Administrators can retrieve custom and complex reports about filesystem content using a very standard language (SQL).
- It can manage a transactionnal stream of events as input (basically, Lustre’s MDT ChangeLog) without loosing events, because it impacts those events in its database using transactions.
- Input management is very modular and many types of information can be processed: namespace scans, streams of events, pre-generated list of entries… Information is processed efficiently, in parallel and asynchronously, thanks to a pipeline architecture.
- Policies are very flexible and easy-to-specify, thanks to its internal Boolean expression engine.
- Large choice of purge triggering: triggers can be specified on global FS usage, OST usage, user or group usage, external command output.
Robinhood FS Monitor v2.2.0 (latest fix: patch2) (source tarball and RPMs)