High Performance Computing

Applications which need the resources of more than one compute node need to use a parallel environment to provide the mechanisms to exchange data between nodes via the network.

On the Nottingham system, all the parallel environments available for programmers are implementations of the Message Passing Interface (MPI) protocol. The same source code can be used in all these environments, though re-compilation is necessary when switching to a different environment. OpenMPI and MVAPICH MPIs are available, and selectable via the modules command. The table below summarises which towers support these MPIs, and which modules are relevant.

	Towers	Modules
OpenMPI	A,B,C,D,E	openmpi/1.3.4-1/gnu
		openmpi/1.3.4-1/intel
		openmpi/1.3.4-1/pgi
MVAPICH	E	mvapich/gcc/2-1.2p1
		mvapich/intel/2-1.2p1
		mvapich/pgi/2-1.2p1

All environments include wrapper scripts for compiling parallel codes, for example mpicc, mpif90 or mpif77. For both OpenMPI and MVAPICH, different modules are needed to select the various compilers.

OpenMPI provides a robust MPI environment for a range of the most popular interconnects used in HPC. Of particular note is the ability of OpenMPI for the Nottingham system to run on either Ethernet or IB using the same binary which can be an added bonus for those people wishing to limit the number of binaries they produce. Thus, users who have produced a binary with the module environment provided under towerE, can run on Towers[A-D]. For example

> module load jupitere
> module load openmpi/1.3.4-1/intel
> module load intel/compiler/111
> make application
> ompisub 4x8 ./application
> module swap jupitere jupitera
> ompisub 16x2 ./application

Another important aspect of openmpi is the ability to optimize the run-time environment to suit a particular applications needs. This is different from other MPIs which make assumptions about the algorithms that may be used and the scale of the parallel job. A full list of the available tuning parameters are available on the OpenMPI website.

MVAPICH runs only over the IB network on towerE, and may be useful for some codes.