A Scalable View-Based Collective I/O Optimization for Large-Scale Parallel Applications

Event Sponsor: 
Mathematics and Computer Science Division
Start Date: 
Oct 10 2008 (All day)
Building/Room: 
Building 221 Conference Room A216
Location: 
Argonne National Laboratory
Speaker(s): 
Francisco Javier Garcia Blas
Speaker(s) Title: 
Carlos III University of Madrid
Host: 
Rob Latham

The aim of this talk is to present the design and implementation of a new file system independent collective I/O optimization based on file views: view-based collective I/O. View-based collective I/O has been implemented and evaluated inside the ROMIO implementation of the MPI-IO standard. The evaluation section shows that view-based I/O outperforms the original two-phase collective I/O from ROMIO in most of the cases for three well-known parallel I/O benchmarks. This is especially due to a smaller cost of scatter/gather operations, a reduction of the metadata overhead, and a smaller number of collective communication and synchronization primitives used in the implementation.

In addition, I will present an implementation of the MPI-IO interface for GPFS inside ROMIO distribution. The experimental section presents a performance comparison among three collective I/O implementations: two-phase I/O, view-based I/O, and a GPFS specific collective I/O implementation based on data shipping. The measurements show that data shipping-based collective I/O performs better for writing, while view-based I/O for reading.

Miscellaneous Information: