RobinHood FS Monitor is a tool for monitoring and purging large filesystems. It is available under the CeCILL-Clicense, which is a French transposition of the GNU LGPL  and is fully LGPL-compatible.

Its main purpose is to collect statistics about file system content and trigger purge when disk usage exceeds a given threshold.

RobinHood is designed to perform all its tasks in parallel, so it is particularly adapted for managing large filesystems with millions of entries and petabytes of data.

Main features

  • Disk space fair-share
  • Policy Engine for temporary filesystems and Lustre-HSM:
    • Purge policies
    • Migration policies
    • Deferred removal policy
    • Directories cleanup policy
  • Attribute-based policies, possibly using sophisticated boolean expressions
  • Accounting and monitoring
  • Custom alerts on filesystem entries
  • Aware of Lustre OSTs and pools

Architecture overview

  • Since version 2.0, Robinhood uses a database engine for managing its list of entries, which offers a lot of benefits:
    • It can manage larger filesystems, because the size of its list is no more limited by the memory of the machine.
    • Purges can be performed immedialty, even after a system crash, because the list is persistent.
    • Robinhood v2 can be split in several daemons running on several nodes, for more scalability: there is no direct communication between several robinhood instances, they are only clients of the same remote database.
    • Administrators can retrieve custom and complex reports about filesystem content using a very standard language (SQL).
    • It can manage a transactionnal stream of events as input (basically, Lustre’s MDT ChangeLog) without loosing events, because it impacts those events in its database using transactions.
  • Input management is very modular and many types of information can be processed: namespace scans, streams of events, pre-generated list of entries… Information is processed efficiently, in parallel and asynchronously, thanks to a pipeline architecture.
  • Policies are very flexible and easy-to-specify, thanks to its internal Boolean expression engine.
  • Large choice of purge triggering: triggers can be specified on global FS usage, OST usage, user or group usage, external command output.

Robinhood FS Monitor v2.2.0 (latest fix: patch2) (source tarball and RPMs)

Post By Gishore J Kallarackal (2,121 Posts)

Gishore J Kallarackal is the founder of techgurulive. The purpose of this site is to share information about free resources that techies can use for reference. You can follow me on the social web, subscribe to the RSS Feed or sign up for the email newsletter for your daily dose of tech tips & tutorials. You can content me via @twitter or e-mail.

Website: → Techgurulive