LosLobos File Systems and Storage

1. Directories
1. Home directories
2. Global scratch directories
3. Local scratch directories
4. /tmp and /usr/tmp directories
2. Permanent file storage
3. Backups and file restoration

1.0 Directories

Your home directory is the default directory you are placed in when you log on to the system. Your home directory is located at /users/<username> where <username> is your login name.

Your home directory is available from each of the LosLobos login or interactive nodes, as well as each of the compute nodes. Because your home directory is not local to any of the compute nodes you will experience degraded job performance if your jobs read or write frequently to your home directory. If possible, your jobs should use local scratch space instead. See Local scratch directories for more information.

You are allocated 500MB of space for your home directory. You should use this space for storing files you want to keep long term such as source code, input data sets, or output data sets. More permanent storage is available. See Permanent file storage for additional details.

To see your disk usage and limits use the quota command:

$ quota
Disk quotas for user janeuser (uid 25124):
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
     /users/    1854064 2000000 2600000            3235       0       0       

Enter "man quota" into a LosLobos prompt to view the man page for the quota command for details about each column of information.

Your home directory is backed up each night. See Backups and file restoration for more information.

1.2 Global scratch directories

Global scratch space is also available from each of the LosLobos interactive and compute nodes at /scratch/<username>. This space is intended for larger, temporary files or files that can be regenerated. Currently there are no quotas in place for the global scratch space, but the space is limited to a total of 500GB for all users.

IMPORTANT: You should consider the global scratch space at /scratch/<username> as volatile; files older than two weeks may be purged. Do not store any files in the global scratch space that you are not prepared to lose. See Permanent file storage for information about more permanent storage for large files.

Like your home directory, global scratch space is not local to the compute nodes. You will experience degraded job performance if your jobs read or write frequently to the global scratch space. If possible, your jobs should use local scratch space instead. See Local scratch directories for how to use local scratch in your work.

1.3 Local scratch directories

Each node has available 7 GB of local scratch space. The space is available only while you are assigned the node through the scheduler (either for a batch job or a scheduled interactive session) and is available at /localscratch.

IMPORTANT: You should consider the local scratch space at /localscratch as extremely volatile. All files are automatically deleted after your job (batch or scheduled interactive) completes and the node is deallocated. All files you want to save must be copied from /localscratch as part of your job. You will not be able to access files in /localscratch after your job has completed.

The local scratch space is local to each compute node, and writing and reading to disk is faster for the local scratch than for your home directory or global scratch. Files in the local scratch space, however, are not available to any other nodes and hence are not directly accessible to your processes running on other nodes as part of your job. Only processes running on the two CPUs which make up a node have direct access to files in the local scratch space.

1.4 /tmp and /usr/tmp directories

The /tmp and /usr/tmp directories are intended for temporary files created during the execution of a job primarily by system processes and third-party software. Your codes should not be programmed to write to these directories, nor should you store files there. The /tmp and /usr/tmp directories may be purged at any time.

back to Top

2.0 Permanent file storage

Alliance users on LosLobos may obtain access to the NCSA UniTree Mass Storage System (MSS). There are no limits to the number or size of files you can store in the mass storage system, and files continue to be available for one year after your account on LosLobos is deactivated. If you are an Alliance user and do not already have an account on NCSA's MSS, contact the Allocations Group at NCSA to obtain access.

Direct access to the NCSA mass storage system from LosLobos is only available using an Alliance digital certificate for authentication. You cannot access UniTree directly from LosLobos using SSH or scp. See the Alliance PKI pages to learn about Alliance digital certificates and how to apply for one. Once you have your Alliance digital certificate see Using GSI Enabled FTP (gsiftp) for details on how to transfer files to and from LosLobos using your certificate.

For specific details about NCSA's mass storage system and the UniTree interface see Mass Storage System -- UniTree at the NCSA website.

back to Top

3.0 Backups and file restoration

Files in your home directory are backed up each night. If you should accidentally delete a file you may contact Alliance consulting to request that the file be restored from the backup. When requesting a file to be restored please include the following information:

• the machine or filesystem (i.e., "LosLobos")
• the full path name of the file that was deleted
• the date and approximate time the file was deleted
• the approximate date when the file was last edited
• the approximate size of the deleted file
• the approximate creation date of the file, or how long the file had existed on the filesystem before being deleted

Obviously files created after the most recent backup cannot be restored. Please also note that the longer ago a file was deleted, the less likely it can be restored.

Files in the global scratchspace /scratch/<username>/ or the local scratch space /localscratch are absolutely not backed up.

back to Top