Siesta

A few notes on the DFT code Siesta and its compilation in TCM.

As ever it is much quicker to compile somewhere on the /scratch disk, even if later one might wish to copy the executable to /rscratch or to one's home directory.

Compiling, serial

Untar the archive.

Make a suitable directory, such as siesta-4.0/serial.

From that directory run sh  ../Src/obj_setup.sh

Copy in a suitable arch.make. The variable part should read something like:

CC = icc
FPP = $(FC) -E -P
FC = ifort
FC_SERIAL = ifort
FFLAGS = -O2 -fPIC

AR = ar
RANLIB = ranlib

SYS = nag

SP_KIND = 4
DP_KIND = 8
KINDS = $(SP_KIND) $(DP_KIND)

LDFLAGS =
COMP_LIBS =
FPPFLAGS = $(DEFS_PREFIX)-DFC_HAVE_ABORT

LIBS = -mkl=sequential

And then type "make". An executable should appear after about three minutes.

Lapack parallelised

The simplest parallelism is just to use OpenMP parallelism on the Lapack and BLAS calls. This is the default for many modern Lapack libraries, so can be achieved simply by changing the last line above to

  LIBS = -mkl

The number of cores used will be the number in the computer at runtime (regardless of how many other processes are running), or the number set by the environment variable OMP_NUM_THREADS.

Rather more of Siesta would be parallelised if one used MPI, and, for that, see below.

MPI parallelised

Siesta claims to need ScaLAPACK and BLACS, and paths to MPI include files and libraries. However, since ScaLAPACK 2, released in 2011, ScaLAPACK has included BLACS, so that is not separately required.

We have separate notes on building ScaLAPACK, simple though it is.

After ScaLAPACK is built, one can return to the Siesta build, and create a new build directory called something like siesta-4.0/parallel. Run obj_setup.sh again, and this time the arch.make file should look something like:

Intel Compilers

CC = icc
FPP = $(FC) -E -P
FC = mpifort
FC_SERIAL = ifort

FFLAGS = -O2 -fPIC

MPI_INTERFACE=libmpi_f90.a
MPI_INCLUDE=.
FPPFLAGS = -DMPI

AR = ar
RANLIB = ranlib
SYS = nag

SP_KIND = 4
DP_KIND = 8
KINDS = $(SP_KIND) $(DP_KIND)

LDFLAGS = 
COMP_LIBS = 

LIBS = /full/path/to/libscalapack.a -mkl=sequential

Gnu compilers

As above, but:

FC=mpgfortran
CC=mpgcc
FC_SERIAL=gfortran

FPPFLAGS=-DGFORTRAN -DFC_HAVE_FLUSH -DFC_HAVE_ABORT  -DMPI

LIBS = /full/path/to/libscalapack.a  -llapack -lblas

Note, in both, the non-empty (but unnecessary) definition of MPI_INCLUDE, and the lack of any -lmpi, for the more modern approach is to use a different compiler name to indicate that one wishes to use MPI, and when that name is used suitable include and library options are implicit. The ScaLAPACK library is best included simply by giving its filename. There is the alternative of

LDFLAGS = -L/full/path/of/directory/containing/libscalapack.a
LIBS = -lscalapack -mkl=sequential

but I rarely find this style clearer.

Do not be tempted by the threaded version of MKL, otherwise there is a danger that on a six core computer one's six MPI processes use six threads each for their lapack calls, and 36 threads at once is too many.