BG/Q File Systems

Vesta and Mira have discrete file systems, with two main user file systems for each machine: home and data/scratch. Both are IBM General Parallel File Systems (GPFS), see: http://en.wikipedia.org/wiki/IBM_General_Parallel_File_System.

The home file system is mounted as /home, and should generally be used for small files and any binaries to be run on the BG/Q. The performance of this file system is reasonable, but using it for intensive I/O from the compute nodes is discouraged because I/O from the compute nodes uses the data file system, which is a fast parallel system and has far more storage space and greater I/O performance than the home directory space.

The Mira home file system is regularly backed up to tape and has snapshot capability.  The data file system is not backed up.  It is your responsibility to ensure that copies of any critical data on the data file system have either been archived to tape or stored elsewhere.

Name Accessible From Type Path Production Backed-up Usage
vesta-home Vesta GPFS /home
or
/gpfs/vesta-home
Yes No general use
projects Vesta GPFS /projects
 
Yes No intensive job output, large files
mira-home Mira
Cetus
Cooley
GPFS /home
or
/gpfs/mira-home
Yes Yes general use
projects Mira
Cetus
Cooley
GPFS /projects Yes No intensive job output, large files

 

Available Directories

Home Directories

  • Created when an allocation (INCITE, DISCRETIONARY, etc.) is granted.
  • Located under /home (mira-home and vesta-home).
  • Each home directory is subject to a quota based on user file ownership. The default quota is 100 GB for Mira and 50GB for Vesta. 

Home Project Directories

  • Available to users by special request.
  • Located under /home/projects.
    • These directories can be requested in order to hold small amounts of code that can be made available to a group of users for sharing.

NOTE: vesta-home/mira-home have user quotas in place, so any files written to a home project directory count against the file owner ’s quota. For quota information, see the Disk Quota page.

Project Directories

  • Created when an allocation (INCITE, DISCRETIONARY, etc.) is granted.
  • Located under /projects.

These project spaces do not have user quotas but a directory quota, meaning that ALL files contained within a project directory, regardless of the username, cannot exceed the disk space allocation granted to the project. For more information on quotas, see the Disk Quota page.