High Performance Computing

Applications that require more than one processor need to be written and compiled in a special way. Teaching multicore programming is beyond the scope of these pages, but links to appropriate resources for both OpenMP and MPI programming are given in the Links section. A forum has also been established for discussion of parallel programming issues.

OpenMPI

To compile or run programs using openmpi, the appropriate environment should be selected via the module commands, for example

module load openmpi/1.3.4-1/intel

The appropriate compiler environment should also be selected if building programs from source:

module load intel/compiler/111

This should ensure the correct versions of the parallel compilers are used; this can be checked by use of the unix 'which' command:

> which mpif90
/opt/openmpi-1.3.4-1/intel/bin/mpif90

A compilation might be as follows:

 mpif90 -O3 -axT -o mpitest mpitest.f
mpitest.f(24): (col.12) remark: LOOP WAS VECTORIZED.
mpitest.f(96): (col.9) remark: LOOP WAS VECTORIZED.
mpitest.f(52): (col.9) remark: LOOP WAS VECTORIZED.

Command line options for the parallel compiler are as for the appropriate standard compiler (ifort in this case). mpif77 is also available, and mpicc is the appropriate command for parallel c programs.

MVAPICH

As for OpenMPI, the appropriate module needs to be selected to build programs using MVAPICH, for example

module load mvapich/intel/2-1.2p1

The compiler modules and commands are the same as for OpenMPI.

OpenMP

Use of OpenMP is more straightforward than MPI, and can be used for programs which can run on one compute node, using up to 8 compute cores on tower E.

Here is a simple example from the tutorial at http: //openmp.org/wp/

       program hello
ccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc 
c Reference:https : //computing.llnl .gov/tutorials/openMP/ c
ccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc
       integer nthreads, tid, omp_get_num_threads,
     + omp_get_thread_num
c fork a team of threads giving them their own copies of variables 
!$omp parallel private(tid)
c obtain and print thread id
       tid = omp_get_thread_num()
       print *, 'hello world from thread = ', tid
c only master thread does this
       if (tid .eq.0) then
       nthreads = omp_get_num_threads()
       print *, 'number of threads = ', nthreads
       end if
c all threads join master thread and disband
!$omp end parallel
       end

Here are basic compile options for Gnu, Intel, and Pgi compilers for Fortran code (C code same - but substitute corresponding C compiler in each case).

gfortran -fopenmp -o hello omphello.f
ifort -openmp -o hello omphello.f
pgf90 -mp -o hello omphello.f