MADNESS on Mira/Cetus/Vesta

Help Desk

Hours: 9:00am-5:00pm CT M-F



MADNESS is a numerical tool kit used to solve integral differential equations using multi-resolution analysis and a low-rank separation representation. MADNESS can solve multi-dimensional equations, currently up to 6D, with a guaranteed precision methodology and should be seen as a multi-level application.

  • First level is the parallel runtime that is able to distribute calculations efficiently on petaflop architectures.
  • Second level is the numerical environment that offers developers a set of numerical tools for advanced scientific computing.
  • Third level MADNESS offers new applications for physics, materials science, and chemistry.

For further information about the code and access to the last version of the source, see the official website M-A-D-N-E-S-S.

The community of MADNESS users and developers provide updates from time to time adding details to help build the code on ALCF machines. For these updates, see Using ALCF.


Basic Building on Blue Gene/Q

Building MADNESS in Blue Gene/Q requires setup of some environmental options so that these options would be incorporated during the configuration process. We suggest the follow options to be set in the shell before running the “configure” script.

export LANG=C
export XL_TOP="/soft/compilers/ibmcmp-nov2012"
export LIBS="${LIBS} -L/soft/libraries/alcf/20120810/xl/LAPACK/lib -llapack"
export LIBS="${LIBS} -L/soft/libraries/essl/current/lib64 -lesslsmpbg"
export LIBS="${LIBS} -L${XL_TOP}/xlf/bg/14.1/bglib64 \
                     -lxlf90_r -lxlfmath -lxlopt -lxl -Wl,-E"
export LIBS="${LIBS} -ldl "
export LIBS="${LIBS} -L${XL_TOP}/xlsmp/bg/3.1/bglib64 -lxlsmp  "
export LIBS="${LIBS} -L/home/avazquez/public -lMADMTXM "
export CPPFLAGS="-D__bgq__"
export CXXFLAGS="-O2 "
export CFLAGS="-O2"
export FFLAGS="-O2 "
export MPICC=mpicc
export MPICXX=mpicxx
export CC=mpicc
export CXX=mpicxx
export F77=mpif77

The configure script should be called out with the follow options:

./configure \
   --enable-debugging=yes \
   --enable-optimization=no \
   --enable-warning=GNU \
   --host=powerpc64-bgq-linux \

At the end of the configuration of MADNESS, running "make" and compiling all the libraries and binaries is now possible.

Running Jobs

In spite of the good performance of MADNESS on a multi-thread architecture, in BG/Q we recommend running with 32 threads per rank, and one rank per node, i.e., mode c1. An example of a queuing job is as follows:

qsub -A MYPROJECT -t 20 -n 128 --mode c1 \
--env MAD_NUM_THREADS=32:MRA_DATA_DIR=/home/avazquez/soft/madnesonly1/src/lib/mra ./moldft

Notice that the number of threads and paths of MRA data is defined in the environmental variables.


  • Older versions of LAPACK 3.3.1 should be avoided because they are not safe for multi-threading.
  • Higher levels of optimization in the compiler may cause race conditions and instability in numerical convergence.
  • It is possible to use a parallel eigensolver from Elemental and replacements of LAPACK for single-node linear algebra routines from Eigen on any platform. These are particularly beneficial on Blue Gene systems due to the relative slowness of the initial guess when using a single-node eigensolver.
  • For additional information regarding MADNESS at ALCF, contact Alvaro VM or Jeff Hammond.