Linux Supercluster Reference Card
This page includes:
For assistance, please contact the Help Desk at help@hpc.unm.edu or (505) 277-8249 (Monday through Friday, 9 a.m. to 5 p.m. Mountain Time)
Depending on your allocation, login to the appropriate Linux Supercluster using secure-shell (ssh):
% ssh roadrunner.alliance.unm.edu
% ssh loslobos.alliance.unm.edu
Transfer files to the Linux Superclusters from your home machine using secure copy protocol (scp):
% scp <file> <user>@<Linux_Supercluster>.alliance.unm.edu:dest
Change password on the Linux Superclusters:
% yppasswd
Change default shell on the Linux Superclusters:
% ypchsh
You need to forward your mail to your local system. To forward mail, create or modify the .forward file with:
<username@host.domain>
If you are interested in debugging your Fortran program, try the NAG compiler first. If you are interested in performance, try all compilers since each are optimized differently. MPI C++ is only supported on Ethernet and may work on Myrinet.
C - GNU
% gcc –o exec [options] filename(s).c
C - PGI
% pgcc –o exec [options] filename(s).c
C++ - GNU
% g++ –o exec [options] filename(s).C
C++ - PGI
% pgCC –o exec [options] filename(s).C
Fortran77 - GNU
% f77 –o exec [options] filename(s).f
Fortran77 - PGI
% pgf77 –o exec [options] filename(s).f
Fortran90/95 - PGI
% pgf90 –o exec [options] filename(s).f90
Fortran90/95 - NAG
% f95 –o exec [options] filename(s).f95
Source code debuggers:
GNU
% gdb
% xxgdb
PGI
% pgdbg
Performance profiler:
PGI
% pgprof
There are several builds of the MPICH MPI library on the Linux Superclusters. Some use the special GM interface to the Myrinet high-speed network hardware and some use TCP/IP. Select an MPICH build by setting the MPIHOME environment variable in your .cshrc file (if your login shell is tcsh). You can set this variable by editing your .cshrc file and removing the comment sentinel (#) from the beginning of the line specifying the MPICH library build that you want to use. Be sure that the beginning of the lines for the other MPICH library builds start with the comment sentinel (#). Refer to your .cshrc file for examples. Log out and log back in to load the new environment. Use the "which mpirun" command to verify the MPICH library build you have chosen.
For example:
% which mpirun
/usr/parallel/mpich-gm.pgi/bin/mpirun
shows that you are using MPICH with the GM interface, and the corresponding MPICH compiling scripts mpif77 and mpif90 will use the PGI Fortran compilers.
C
% mpicc –o exec [options] filename(s).c
C++
% mpiCC –o exec [options] filename(s).C
Fortran77
% mpif77 –o exec [options] filename(s).f
Fortran90
% mpif90 -o exec [options] filename(s).f90
PBS supports two modes for running jobs – batch and
interactive. With batch mode, a user submits a job, and the job is queued, scheduled
and run without any further interaction. With interactive mode, a user
submits a request for an interactive session on a set of compute nodes assigned
by the PBS job manager.
The following examples show simple scripts for batch mode jobs.
Portable Batch System (PBS)
PBS Batch Mode Sample Scripts:
Request 16 nodes and start one mpi process on each node:
#PBS -l nodes=16:ppn=2
mpirun -machinefile $PBS_NODEFILE -np 16 myprog.exe
Request 8 nodes and start 2 mpi processes on each node:
#PBS -l nodes=8:ppn=2
mpirun -machinefile $PBS_NODEFILE -np 16 myprog.exe
Standard output and error files will be written to your working directory. To submit the script to PBS, use the following command:
% qsub <pbs_script_file>
PBS Interactive Mode Sample Session:
Request an interactive session with 8 nodes:
% qsub –I –l nodes=8
Run your program as follows:
Start one mpi process on each node:
% mpirun -machinefile $PBS_NODEFILE -np 8
Start two mpi processes on each node:
% mpirun -machinefile $PBS_NODEFILE -np 16 myprog.exe
Release your nodes when finished by typing:
% exit
Display contents and status of queue:
% qstat -a
Remove job from queue:
% qdel <job_id>