Vampir

References

Vampir manual - https://www.vampir.eu/tutorial/manual

 

Introduction

Vampir is a GUI interface for browsing traces collected from applications using the Score-P instrumentation library. The Vampir ecosystem consists of several parts

  • Score-P - an instrumentation framework and library used to collect profile and trace information while your code runs
  • VampirServer - a backend for the Vampir GUI that reads and processes trace files
  • Vampir GUI - a GUI that runs on a local machine and connects to the Vampir server

These components are installed and run in different places. The Score-P libary is installed on Mira and Cetus and can be used to collect data for applications run on those machines. The VampirServer is installed and can be run on Cooley and can be used to view trace files collected on Mira and Cetus. The Vampir GUI is installed on Cooley or may be installed on the users local machine.

 

Installation on the ALCF System

The Vampir components are installed on Mira, Cetus, and Cooley in the directory

/soft/perftools/vampir

 

Environment Setup

Add the softkey

+vampir

to your ~/.soft or ~/.soft.cooley files and run the command resoft

 

Instrumenting with Score-P

Follow the directions above to add Vampir to your environment.

Score-P provides compiler wrappers that will add the necessary instrumentation to your code. To utilize the wrappers simply preface all of your compiler and linker commands with scorep.

scorep mpixlc  ....

scorep mpixlf ....

scorep mpicc ....

 

Running with Score-P

After instrumenting with Score-P several environment variables are used to control the data collection. For the first run with Score-P it is recommended to collect only profile data by adding the following environment variable to your qsub submission

qsub ... --env SCOREP_ENABLE_TRACING=false:SCOREP_ENABLE_PROFILING=true:SCOREP_TOTAL_MEMORY=100M ....

Upon job completion a directory will be created containing the Score-P collected data. The directory name will start with scorep-

The collected profile information may be examined using the command scorep-score

scorep-score {scorep-data-dir}/profile.cubex

After reviewing the profile information re-run with tracing enabled

qsub ... --env SCOREP_ENABLE_TRACING=true:SCOREP_ENABLE_PROFILING=false:SCOREP_TOTAL_MEMORY=100M ....

This run will produce a much larger volume of tracing data that can be examined using the VampirServer and Vampir GUI

 

Setting up the Vampir GUI client

It is recommended to install the Vampir GUI client on your local machine for best performance, alternatively it may be run under X-11 on Cooley.

To install Vampir locally the appropriate Linux, Windows, or Mac Vampir clients may be downloaded from Mira, Cetus, or Cooley from the directory

/soft/perftools/vampir/downloads

In addition to the client version above the following license should be downloaded if installing Vampir locally

/soft/perftools/vampir/license/vampir-remote.license

and imported into the GUI when it is started up.

 

Starting the Vampir Server on Cooley 

Configure the Vampir environment on Cooley by adding +vampir to your ~/.soft.cooleyfile and running 'resoft' .  Best performance for the Vampir Server will be obtained by running the server on the Cooley compute nodes by using the command -

vampirserver start -t <wait-time> -n <numtasks> -- -A <project> -w <minutes>

wait-time - timeout limit for vampirserver to start

numtasks - number of Vampir analysis tasks, recommended values are multiple of the node core count such as 12, 24, 36, ... on Cooley

project - project to charge for qsub submissions

minutes - number of minutes to reserve the computes nodes for use by the Vampir Server

Alternatively for small trace analysis the Vampir Server may be run on the Cooley  login node via the command

vampirserver start smp

In either case the output should look similar to

Launching VampirServer...
VampirServer 8.4.1  (r9456)
Licensed to Argonne NL
Running 32 analysis processes... (abort with vampirserver stop 22352)
VampirServer <22352> listens on: vs47:30087

where the last line indicates the node name (vs47) and port number (30087) where the server is listening for a connection from the GUI client

When finished with Vampir you must terminate the Vampir Server with the command

vampirserver stop <id-number>

Where <id-number> is the value printed at server startup as "VampirServer <id-number> listens on: vs47:30087"

Connecting the GUI to the Server

Once the Vampirserver has been started and produced output indicating where it is listening the Vampir client can be connected to view the trace data.

For locally installed Vampir clients create a new ssh tunnel to the server using the command

ssh -L 30001:vs47:30087 <user>@cooley.alcf.anl.gov 

Note the node name (vs47) and port number (30087) used in the example above are the values reported by the Vampir Server at startup. These values may be different for different Vampir Server launches - please replace them with the values reported at Vampir Server launch 

Launch the Vampir GUI on your local machine and click on 'Open Other' then select 'Remote File' and set

Server: localhost

Port: 30001

Authentication: none

Connection type: Socket

then select 'connect'. Use the GUI to navigate to the directory containing the trace files and then select the traces.otf2 file to open. This will open the trace files for viewing, depending on the size of the trace files and the speed of the connections opening the files for viewing can require a little time.

When using the Vampir client installed on Cetus it is not necessary to create an SSH tunnel, instead follow the steps above to start the GUI and use the server and port printed when the Vampir server was started.