New User Guide

USER TIP: View "Getting Started" --a video introduction to ALCF services and resources with useful tips to boost your job throughput.

Step 1. Request an ALCF Project

Step 2. Get an ALCF User Account

Step 3. Logging in to an ALCF Resource

Step 4. Setting Up Your Computing Environment

Step 5. Data

Step 6. How to Run a Job

Step 7. Managing Your ALCF Account

Step 8. ALCF Acknowledgement Policy

Step 9. Getting Additional Support


Step 1. Request an ALCF Project

You must get a project on the system you will be using before you can proceed. If you do not have a project, please visit to establish one.

Step 2. Get an ALCF User Account

In order to use the resources at ALCF, users need to have an account on our systems. The following steps provide some guidelines on how to request for a user account at the ALCF.

  • Please go to and click on 'Request an account'.
  • Click 'Proceed with account request'.
  • Select the appropriate radio button, input your email address, then click 'Proceed to next step'.
  • When prompted to search for an existing account, please enter the relevant information, then hit 'search' to look for an existing user account.
  • If no existing user account is found, hit 'Continue with Account Request'.
  • If you find an existing user account that needs to be reactivated, please fill out the form on the page, then hit the 'Submit Reactivation Request'.
  • From the 'Project Information' page select the project that you would like to be a member of, then scroll to the bottom of the page, and click 'Proceed to next step'.
  • Follow the steps outlined within the account request form, making sure ALL personal information is filled in thoroughly and accurately.
    • Please include any alternate addresses, phone numbers and email addresses (home etc.)
    • Proceed to the next page by choosing the appropriate button ('I am a US citizen', 'I am NOT a US citizen')
  • When you get to the page that specifies 'Type of Account'.
    • Be sure to select 'Annual'
    • Your project's PI name should be in the 'Sponsor Full Name' field. If you are the PI, this will be the name of the ALCF staff member that you are in contact with.
    • Your project PI's email address should be in the 'Sponsor Email Address' field. If you are the PI, then please enter the email address of the ALCF staff member you have provided above.
    • Click 'Proceed to next step'
  • Select the appropriate resource based on the project you selected earlier.
  • Read through the Argonne National Lab Computer User Agreement, then click 'Submit'

Step 3. Logging into an ALCF Resource

The logins, also known as the front-end nodes, are the nodes that users access through the ssh command to the ALCF systems. These nodes allow for interactive activities such as editing files and compiling. The compute nodes are not directly accessible to the user, but are where users code is executed when submitted with the qsub command. An important aspect of the Blue Gene system to note is that the hardware and operating systems of the login and compute nodes is different.

Activating Your Cryptocard

  • After you have received your Cryptocard token and before you can use it, you must call the ALCF Service Desk (630-252-3111 or 866-508-9181) for us to verify your identity and activate the token. If you do not perform this step, you will not be able to log on to ALCF resources using your Cryptocard token.

Logging In Using Your Cryptocard

  • To log in to an ALCF resource, for example Vesta, from a Unix machine, ssh to ssh If your username on your local machine is different from your username on ALCF resources, you will need specify your ALCF account username within the SSH command. This can be done in one of two ways: by using the -l option or by prepending your username to the hostname.
    • ssh -l <alcf_username> vesta
    • ssh <alcf_username>
  • When prompted for a password, the user must provide the PIN as well as the one-time password (Cryptocard password obtained by pressing the button on your token) to authenticate.

If you have any problems logging in using the Cryptocard please refer to the help section for troubleshooting my cryptocard or call the ALCF Service Desk (630-252-3111 or 866-508-9181).

Step 4. Setting Up Your Computing Environment

Setting Up Your Shell

Once you are logged into the BG/Q system, you will be sitting at a UNIX shell prompt in your home ($HOME) directory. If you have never used UNIX before, you will need to learn some basic commands before you will be able to submit jobs. A number of links to tutorial information may be found at the Unix Guru Universe Beginners' Pages (external link). Your account will be set up with the default shell bash, unless you requested a different shell in your account request.

We also provide tcsh and zsh, You can change your shell by logging in to your account web page. Scroll down to the "Unix Shell" section and choose your new shell.

Setting Up Your Software Environment

A software environment management system called SoftEnv is used to control system path, and other environment variables required to run application software on the BG/Q. The first time a user logs on to BG/Q, a '.soft' configuration file will automatically be created in the user's home directory. This file will be set up with the default applications environment -- typically there will be a single line with '@default' in the file. Depending on the user's applications, further modifications to the .soft file may be necessary.

SoftEnv man pages are available in the default environment (use 'man softenv'). The command 'softenv' will list all available applications. Users wishing to gain a more complete understanding of how softenv works may read the complete softenv documentation.

Step 5. Data

Transfer Utilities

sftp and scp

These standard utilities are available for local area transfers of small files.

Data Transfer Service

Globus Online: the ALCF Globus Online Endpoint for Mira, Cetus and Tukey is alcf#dtn_mira. See more information about this service at the following page: Using Globus Online.

Step 6. How to Run a Job

For extensive information on running and queuing jobs, please visit our web pages for the BG/Q systems or BG/P systems. Here's some basic information to get you started.

Basic Job Submission

Direct executable, no script

qsub -A <project> -n <nodes> -t <walltime> --mode <mode> --env "FOO=a:BAR=b" ./<exe>

<project> - Assigned project short name

<nodes> - Number of nodes to use; will be rounded to fit in the nearest partition size.

<walltime> - Maximum wall clock time to allow your job to run, specified in minutes

<mode> - Number of ranks per node, c1 - 1 rank per node, c8 - 8 ranks per node. Valid values: c1, c2, c4, c8, c16, c32, c64.

<exe> - binary executable.

Submitting a script

Normally, a Cobalt job submission directly specifies an executable, rather than a shell script as with other queuing systems. However, Cobalt can also handle scripts. Within your script, use the command runjob to start an execution.

qsub -A <project> -n <nodes> -t <walltime> --mode script --env "FOO=a:BAR=b" ./

Here is an example script:

  echo "Starting Cobalt job script"
  # see runjob --help for more options
  runjob --block $COBALT_PARTNAME --np $(($COBALT_JOBSIZE*16)) --ranks-per-node 16 \
     --verbose 2 --envs FOO=$FOO --envs BAR=$BAR : myprogram.exe myprogarg

Step 7. Managing Your ALCF Account

Updating Your Contact Information

In order to communicate effectively with users and to abide by DOE regulations, we must have the most current information for each user. Please keep your personal information up to date. This is important so that we can contact you.

To update your information, please login to the ALCF accounts page. If you forget your password, please call the ALCF Service Desk at 866-508-9181. Once your identity is verified, the password will be reset and provided.

Step 8. ALCF Acknowledgement Policy

As an active Argonne Leadership Computing Facility (ALCF) user conducting research on an ALCF resource, we request your continued cooperation and compliance with the work acknowledgment policy pertaining to your computing allocation award. For guidance on acknowledgements, please refer to this policy page on our website:

Step 9. Getting Additional Support

Training Opportunities

ALCF conducts workshops and other events to train new users, help scale existing projects, and introduce new techniques to experienced users. Click to view upcoming training events.

Contacting Us

ALCF Service Desk staff is available from 9:00 am through 5:00 pm (US Central Time), Monday through Friday for phone-based support. Email support is the preferred method of requesting assistance and covers standard hours (9:00 - 5:00) as well as emergency response for after-hours and weekend support.