The growing demand for scalable storage I/O is evident from the use of large compute clusters for a burgeoning set of applications that require highly scalable and parallel execution. Much of the research in storage systems has focused on scaling the "data path", i.e. scaling the performance for file data operations. However, the same cannot be said about file metadata scaling, which is a growing concern for storage system users and vendors. I will present our ongoing work on building a metadata service, in form of new directory subsystem called GIGA+, that can scale both capacity (i.e., store billions to trillions of files in a directory) and performance (i.e., handle more than 100K operations per second).

GIGA+ uses an indexing technique that embodies two principles -- high decentralization and high parallelism -- for both indexing and accessing the metadata. Our technique partitions a directory over a scalable number of servers in an incremental, load-balanced, and unsynchronized manner. We achieve highly parallel growth by allowing every server to grow its partitions independently, without synchronizing with the rest of the system. While accessing the directory, our approach tolerates the use of stale partition-to-server mapping at the clients without affecting the correctness of the system. Currently, we are building a prototype of our directory subsystem in Parallel Virtual File System (PVFS), an open-source cluster file system.

This is joint work with Garth Gibson (CMU), Milo Polte (CMU) and Sam Lang (Argonne)