Linux Supercluster Reference Card

This page includes:

For assistance, please contact the Help Desk at help@ahpcc.unm.edu or (505)  277-8249 (Monday through Friday, 9 a.m. to 5 p.m. Mountain Time)

For additional documentation, please see: http://www.ncsa.uiuc.edu/SCD/Info/MoreHelp.html

Depending on your allocation, login to the appropriate Linux Supercluster  using secure-shell (ssh):

%  ssh roadrunner.alliance.unm.edu

%  ssh loslobos.alliance.unm.edu

Transfer files to the Linux Superclusters from your home machine  using secure copy protocol (scp):

%  scp <file> <user>@<Linux_Superluster>.alliance.unm.edu:dest

Change password on the Linux Superclusters:

%  yppasswd

Change default shell on the Linux Superclusters:

%  ypchsh

You need to forward your mail to your local system. To forward mail,  create or modify the .forward file with:

<username@host.domain>

If you are interested in debugging your Fortran program, try the NAG  compiler first. If you are interested in performance, try all compilers since each are optimized differently. MPI C++ is only supported on Ethernet and may work on Myrinet.

C - GNU

%  gcc –o exec [options] filename(s).c

C - PGI

%  pgcc –o exec [options] filename(s).c

C++ - GNU

%  g++ –o exec [options] filename(s).C

C++ - PGI

%  pgCC –o exec [options] filename(s).C

Fortran77 - GNU

%  f77 –o exec [options] filename(s).f

Fortran77 - PGI

%  pgf77 –o exec [options] filename(s).f

Fortran90/95 - PGI

%  pgf90 –o exec [options] filename(s).f90

%  pghpf –o exec [options] filename(s).hpf

Fortran90/95 - NAG

%  f95 –o exec [options] filename(s).f95

Fortran90/95 - Absoft

%  f90 –YEXT_NAMES=LCS –YEXT_SFX=_ –o exec [options] filename(s).hpf

Source code debuggers:

GNU

%  gdb

%  xxgdb

PGI

%  pgdbg

Performance profiler:

PGI

%  pgprof

TotalView MPI Debugger:

Etnus

%  totalview

For example:

%  which mpicc

/usr/parallel/ll/mpich-gm.pgi/bin/mpicc

shows that you are using GM with PGI compilers.

C

%  mpicc –o exec [options] filename(s).c

C++

%  mpiCC –o exec [options] filename(s).C

Fortran77

%  mpif77 –o exec [options] filename(s).f

Fortran90

%  mpif90 -o exec [options] filename(s).f90

PBS supports two types of modes for running your job – batch  and interactive. With interactive, a session is automatically started  on one of the compute nodes assigned to you through the Job Manager. In  the following scripts, ppn = process(es) per node and n = number of nodes requested.

Portable Batch System (PBS)

PBS Batch Mode Sample Scripts:

Myrinet - 1 ppn

#PBS –l nodes=n
gmpiconf $PBS_NODEFILE
mpirun -np n.exec

Ethernet - 1 ppn

#PBS  –l nodes=n
mpirun  -np n.exec

Myrinet - 2 ppn

#PBS  –l nodes=n
gmpiconf2  $PBS_NODEFILE
mpirun  –np 2*n <prog>.exec

Ethernet - 2 ppn

#PBS  –l nodes=n
mpirun -np 2*n -machinefile $PBS_NODEFILE <prog>.exec

Standard output and error files will be written to your working directory.  To submit the script to PBS, use the following command:

% qsub <pbs_script>

PBS Interactive Mode Sample Session:
Request an interactive session with n nodes:

% qsub –I –l nodes=n <pbs_script>

Change directory to:

$PBS_O_WORKDIR (your working directory).

Run your program as follows:

MyrinetEthernet1 ppn

If Myrinet 1 ppn:

% gmpiconf $PBS_NODEFILE
% mpirun  -np n.exec

If Ethernet 1 ppn

% mpirun  -np n.exec

If Myrinet 2 ppn:

% gmpiconf2 $PBS_NODEFILE
% mpirun –np 2*n <prog>.exec

If Ethernet 2 ppn

% mpirun –np 2*n <prog>.exec

Release your nodes when finished by typing:

% exit

Display contents and status of queue:

% qstat

Remove job from queue:

% qdel <job_id>