Scalasca

Introduction

Scalasca is a software tool that supports the performance optimization of MPI and OpenMP parallel programs by measuring and analyzing their runtime behavior. The analysis identifies potential performance bottlenecks – in particular those concerning communication and synchronization – and offers guidance in exploring their causes.

Location

To add Scalasca to your environment use the softenv key "+scalasca"

Scalasca is installed in: /soft/perftools/scalasca

The Cube viewer is installed : /soft/perftools/cube
 

Using Scalasca

0. Usage

Run the scalasca command (or its sub-commands) with no arguments for usage information:

scalasca

1. Instrumentation

Prepare application objects and executable for measurement by prefixing compile and link commands:

scalasca -instrument  mpixlf77 -O3 -c bt.f
scalasca -instrument  mpixlf77 -O3 -o bt.1024 *.o

or as a shortcut the same commands can run as:

skin mpixlf77 -O3 -c bt.f
skin mpixlf77 -O3 -o bt.1024 *.o

2. Analysis

Run instrumented application under control of measurement system by prefixing MPI launch commands:

scalasca -analyze  qsub -A testproject -n 16 --mode c16 ./bt.1024

or alternatively

scan qsub -A testproject -n 16 --mode c16 ./bt.1024

 

For scripts jobs the Scalaca commands should prefix the runjob commands:

scalasca -analyze  runjob --np 1024 --ranks-per-node 16 : ./bt.1024
scan runjob --np 1024 --ranks-per-node 16 : ./bt.1024

Hardware counters can be included in measurements via '-m' flag or EPK_METRICS environment variable, e.g.:

EPK_METRICS=PAPI_FP_OPS:PEVT_IU_IS1_STALL_CYC

The '-t' flag used to specify trace collection and automatic parallel analysis, where event data for each process and thread written to separate files (by default).

 

3. Examination

Interactively explore the generated measurement analysis report:

scalasca -examine  scorep_bt_16p1024_sum

or

square  scorep_bt_16p1024_sum

Alternatively a short summary of the collected performance data can be viewed by running

scorep-score -r scorep_bt_16p1024_sum/profile.cubex

Additional Information

Additional information may be found on the Scalasca website - www.scalasca.org

Consult the User Guide for full usage information or the Quick Reference for a brief summary.

Known Issues on BG/Q

  • The parallel trace analyzer scout.hyb generally coredumps after completing trace analysis
  • SCAN options are automatically passed through Cobalt qsub and set as necessary for runjob, however, other EPIK environment variables (e.g., ELG_SION_FILES) are not.
    • Additional EPIK environment variables are best specified via an ./EPIK.CONF file (in the working directory)
  • Shared node-level counters are read and stored for each process/thread