Job Scheduling Policies on Cooley

There are two primary queues on Cooley, default and debug

The default queue is for production use, and is the default queue for jobs that are submitted without a queue specified.  It has the following characteristics:

  • Max. runtime: 12 hours
  • Max. job size: 110 nodes (the other sixteen nodes are dedicated to debugging)
  • Max. running jobs per user: 10
  • Max. node-hours (queued and running): 1320
  • Priority: FIFO -- (jobs are run in order, with small, short jobs run on any otherwise-free nodes)

In addition, there are sixteen nodes set aside for dedicated debugging in the debug queue.  This is intended for short debugging and interactive visualization runs only.  It has the following scheduling policy:

  • Max. runtime: 2 hour
  • Max. job size: 16 nodes
  • Max. running jobs per user: 1
  • Priority: FIFO -- (jobs are run in order, with small, short jobs run on any otherwise-free nodes)

For jobs that require public network connectivity on the compute nodes (i.e. connectivity to non-ALCF resources), you may include the argument --attrs=pubnet in your qsub command . 

For jobs that use the GPUs directly for computation (e.g. CUDA) and don't require an X sever, you may wish to include the argument --attrs=nox11 in your qsub command.  This will stop the X server that normally runs on the nodes in order to prevent any performance impact on your GPU jobs.

If required, the above job attributes may be combined as a colon-separated list, e.g. --attrs=pubnet:nox11

While we currently continue to maintain special-purpose queues for the above functions (the queues named pubnet, pubnet-debug, nox11, and pubnet-nox11) in order to maintain compatibility for submission scripts that use them, these queues have been deprecated in favor of using the above job attributes, which provide more flexbility and can be used within reservations.

If you have needs not addressed by the standard queues, please send mail to support@alcf.anl.gov requesting a reservation.

We will monitor Cooley's queues and evaluate the above policies as needed. Your feedback (send e-mail to support@alcf.anl.gov) is appreciated.