Data and Networking

Data Storage

The ALCF’s data storage system is used to retain the data generated by simulations and visualizations. Disk storage provides intermediate-term storage for active projects, offering a means to access, analyze, and share simulation results. Tape storage is used to archive data from completed projects.

Disk Storage: The Blue Gene/Q data systems consist of 384 I/O nodes that connect to 16 storage area networks (SANs) that control 8,960 disk drives with a total capacity of 28.8 PB of raw storage and a maximum aggregate transfer speed of 240 GB/s. The ALCF uses the GPFS file system to access the storage.

Tape Storage: ALCF computing resources share two 10,000-slot libraries using LTO4 tape technology. The LTO tape drives have built-in hardware compression with compression ratios typically between 1.25:1 and 2:1, depending on the data, giving an effective capacity of 16-24 PB.

Networking

Networking is the fabric that ties all of the ALCF’s computing systems together. The Blue Gene/Q systems have an internal proprietary network for communicating between nodes. InfiniBand enables communication between the I/O nodes and the storage system. Ethernet is used for external user access, and for maintenance and management of the systems.

The ALCF’s Blue Gene/Q systems connect to other research institutions using a total of 100 Gb/s of public network connectivity. Scientists can transfer datasets to and from other institutions over fast research networks such as the Energy Science Network (ESnet) and Internet2.