|
Tips on Running Parallel Jobs with
LoadLeveler
- For any jobs which intend to use the high performance switch and User
Space protocol for communications, you will need to specify both of
the following in order to insure running in User Space mode:
#@ requirements = (Adapter == "hps_user")
#@ environment = MP_EUILIB=us
- If you have multiple #@ requirements statements, only the last will
have effect. If you need to specify multiple requirements, group them
with the proper syntax. For example:
#@ requirements = (Adapter == "hps_user")
&& (Memory >= 256)
- If you have multiple #@ environment statements, only the last will
have effect. If you need to specify multiple environment variables,
separate them by semi-colons with a single #@ environment statment.
For example:
#@ environment = MP_EUILIB=us;MP_INFOLEVEL=3;MP_LABELIO=yes
- Be careful about setting MP_EUILIB in your .cshrc or .profile file.
This will override whatever you specify in your LoadLeveler command
script. If your communications throughput does not come close to the
figures below, it is probably because of this.
- Communications throughput - rough estimates. You should expect something
close to these for jobs with message sizes greater than 500,000 bytes
in length.
IP over ethernet = < 1 MB/sec
IP over switch = 9-12 MB/sec
US over switch = 30-34 MB/sec
- LoadLeveler will "source" both your .cshrc and .login files.
C Shell users may wish to exclude LoadLeveler from running interactive
only commands in these files by doing something like:
if ($?prompt) then
setenv TERM vt100
set filec
set prompt = "`hostname -s`% "
setenv MP_EUILIB us
:
:
endif
- Do not use the #@ executable statement if you are running parallel
jobs. Parallel jobs use the job command file as the executable.
- Use your FULL email address if you specify "notify_user"
in your command script.
- Do not try to use LoadLeveler macros, such as $(job_name), $(cluster)
or $(process) as script variables. They will not be recognized by the
shell.
|
