Modern architectures such as BlueGene and environments such as computational grids offer exciting possibilities for developing dramatically new classes of applications. Despite their appeal, optimizing applications to run efficiently in these complex environments is often a significant challenge. For example, the underlying communication infrastructure may be non-uniform, allowing neighboring portions of a parallel computation to communicate at low cost with communication involving distant portions being more expensive. Tuning an application to reflect the architecture can produce significant performance gains, but at the same time runs the risk of making the application less portable due to encapsulating environment-specific knowledge into the application. A more appropriate place for these types of optimizations is within a runtime system (middleware) layer. Further, more dynamic runtime systems can couple knowledge of the running application with knowledge of the underlying environment to optimize application performance during execution. Some of these optimizations may not even be possible for a human using a priori information.

This talk describes experiences with using the Charm++ and Adaptive MPI runtime systems to optimize application performance in challenging environments. Specific examples are taken from a long-term project that has developed techniques for deploying tightly-coupled parallel applications in grid computing environments. Because the techniques are implemented at the runtime system level, they are available to applications that use a variety of problem decomposition styles and require little or no modifications to application software.