Low latency, high bandwidth interconnects are an important component of High Performance Computing (HPC) systems, and Remote Direct Memory Access (RDMA) has become the most popular commercially-available technology for realizing these interconnects.  There are currently three RDMA families -- InfiniBand, iWARP, and RoCE.  As of June, 2012, 209 (41.8%) of the top 500 Super Computer systems used InfiniBand as their interconnect.  Recently interest in RDMA has grown outside the HPC community, into financial and commercial data centers, and into general bulk-data movement applications such as GridFTP.

This talk describes some of our current work applying RDMA in two areas: data movement over distance, based on some experiments with RoCE on the Advanced Networking Initiative (ANI) 100G Testbed; and development of EXtended Sockets (EXS) as a high-level API for convenient, efficient access to RDMA in domains where existing APIs, such as MPI, are not necessarily appropriate.