NWCHEM is a parallel quantum chemistry mainly written in Fortran77 and uses MPI and OpenMP for distributed and multicore computing. NWCHEM was designed to solve large-scale electronic structure calculations with Hartree-Fock, Density Functional Theory and other wavefunction correlated methods, see full features in nwchem-sw.org.
Using NWCHEM at ALCF
ALCF provides binaries and compiling instructions for NWCHEM. For questions, contact us at support@alcf.anl.gov.
How to Obtain the Code
NWCHEM is an open-source code the official web site ishttp://www.nwchem-sw.org . Change log and release announcements could be found here
http://www.nwchem-sw.org/index.php/Download.
Official Nwchem source could be downloaded from the project’s github.com repository https://github.com/nwchemgit/nwchem/releases. For detailed compiling options, please visit the wiki page of Nwchem, https://github.com/nwchemgit/nwchem/wiki/Compiling-NWChem. The follow instructions were tested in for the version 6.8.1.
cat nwchem_envs_681.sh export NWCHEM_TOP=/projects/nwchem/nwchem-6.8.1 export NWCHEM_TARGET=LINUX64 export USE_MPI=y export USE_CPPRESERVE=y export NWCHEM_MODULES="all" export USE_MPI=y export USE_MPIF=y export USE_MPIF4=y export USE_OPENMP=1 export USE_KNL=1 export USE_OPENMP=y export INTEL_64ALIGN=1 export USE_NOIO=1 export USE_GAGITHUB=1 export CRAYPE_LINK_TYPE=dynamic export export ARMCI_NETWORK=MPI_TS export BLAS_SIZE=4 export BLASOPT=" -lmkl_intel_lp64 -lmkl_core -lmkl_intel_thread -lpthread -lm -ldl" export SCALAPACK=" -lmkl_scalapack_lp64 -lmkl_intel_lp64 -lmkl_core -lmkl_intel_thread -lmkl_blacs_intelmpi_lp64 -lpthread -liomp5 -ldl" export BLAS_LIB=$BLASOPT export LAPACK_LIB=$BLASOPT export SCALAPACK_SIZE=4 export USE_KNL=1 export USE_64TO32=1 cd $NWCHEM_TOP/src make nwchem_config make 64_to_32 make
Alternatively, binaries can be found in the folders under /soft/applications/nwchem
Running Jobs on Theta
The following script “submit.sh” is an example to run Nwchem in Theta a job of 8 nodes with 64 MPI ranks per node. The job can be submitted with command “qsub submit.sh”
cat submit.sh #!/bin/bash #COBALT -n 8 #COBALT -t 30 #COBALT -A myproject module add atp bin=/soft/applications/nwchem/6.8/bin/nwchem echo "Running Cobalt Job $COBALT_JOBID." rpn=64 df -k export MPICH_GNI_MAX_EAGER_MSG_SIZE=16384 export MPICH_GNI_MAX_VSHORT_MSG_SIZE=10000 export MPICH_GNI_MAX_EAGER_MSG_SIZE=131072 export MPICH_GNI_NUM_BUFS=300 export MPICH_GNI_NDREG_MAXSIZE=16777216 export MPICH_GNI_MBOX_PLACEMENT=nic export MPICH_GNI_LMT_PATH=disabled export COMEX_MAX_NB_OUTSTANDING=6 export LD_LIBRARY_PATH=/opt/intel/compilers_and_libraries_2018.0.128/linux/compiler/lib/intel64_lin aprun -n $((COBALT_JOBSIZE*rpn)) -cc depth -d1 -j1 $bin test.nw qsub submit.sh