Parallel File Systems: From Research to Deployment

Philip Carns
Seminar

The growing scale of modern HPC deployments has intensified the demand for throughput from parallel file systems. However, traditional parallel file system designs face scalability challenges as the number of I/O servers is increased in order to keep pace with throughput requirements. This is particularly evident in file system metadata and management operations. This seminar presents techniques for addressing some of these scalability challenges within a parallel file system through the use of intelligent servers and collective communication. These techniques have been prototyped and evaluated using the PVFS file system.

This seminar will also relate experiences in deploying PVFS in a large scale commercial production environment. Prominent issues in this environment include high availability, rapid problem analysis, consistent management of long term environments, and replication.