Today’s leadership computing facilities have enabled the execution of transformative simulations at unprecedented scales. As the computing power of these supercomputers continues to increase, so does the size of the datasets they produce for researchers to analyze and produce insights. However, the development of I/O and storage capabilities of these supercomputers to handle these larger datasets is not keeping pace with the leaps being made in compute power. This imbalance has caused I/O to become a significant bottleneck for leadership computing applications, slowing the pace of scientific discovery that is possible with modern supercomputers. Simulation-time analysis can help overcome this by performing the analysis tasks while the simulation is executing. In this talk, we present our work [1] in scheduling of in-situ analysis as a numerical optimization problem to maximize the number of online analyses subject to resource constraints such as I/O bandwidth, network bandwidth, rate  of computation and available memory. We demonstrate the effectiveness of our approach using two application-driven case studies on the Mira IBM Blue Gene/Q (BG/Q) supercomputing system.

In the second part of the talk, we will present effective techniques to improve the data movement of independent parallel I/O on the BG/Q system. Independent I/O is one of the common techniques employed in supercomputers for reading and writing data from storage. We will discuss a system interconnect route-aware and load-aware algorithm to modify the existing  Blue Gene/Q (BG/Q) supercomputer's bridge node assignment. This results in a different path taken by the I/O messages to reach the I/O nodes. Our algorithm routes 1.4X fewer messages through the bridge nodes which connect to the I/O nodes on the BG/Q. We reduce the network contention and reduce the write time by an average of 60% over the default independent I/O and by 20% over collective I/O on up to 8192 nodes on the Mira BG/Q system for scientific application I/O.  

[1] Preeti Malakar, Venkatram Vishwanath, Todd Munson, Christopher Knight, Mark Hereld, Sven Leyffer, and Michael E. Papka,"Optimal scheduling of in-situ analysis for large-scale scientific simulations" In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC ’15),November 2015, Austin, Texas, USA.