Using the tg-policy Command
Home > User Info > Using the tg-policy Command
NOTE: The tg-policy command is not currently available on the following systems: Cobalt (NCSA), Linux Cluster (Purdue), and DataStar (SDSC).
For detailed information on Data, Scheduling, and Filesystem policies at each resource, use the tg-policy command. The following flag options are available:
-data Data Policy
-sched Scheduling & Queue Policy
-fs Filesystem summary
-help Help display
-v tg-policy command version
-a return all results
Sample output from tg-policy -a:
ANL IA-64
Scheduling Policy:
-------------------------- General Policy:
---------------
Given the small node count at this site please submit jobs with extra
discretion. Select small node counts for long running jobs, or short
execution times for large node count jobs. Priority is given to the
realtime visualization systems between 8am-6pm. Jobs will be interrupted
if someone requires physical access to the visualization resources.
Advance notice may or may not be provided. Specific Visualization Policy:
------------------------------
Visualization work has the highest priority of all jobs in progress,
waiting or queued on the system at any time. There are two visualization
categories available: Interactive Visualization and Batch Visualization.
Interactive Visualization work using directly attached physical devices
to local nodes receive the highest priority available. Further, all
visualization jobs may and will pre-empt jobs in progress, queued, or
waiting. Interactive visualization jobs, of any type, take priority
over batch visualization. Cross Site Reservations:
------------------------
Cross site reservations are a higher priority than local reservations.
However, interactive visualization jobs using locally attached physical
devices still have a higher priority than even cross-site reservations. Changes in policy will be published here through the "tg-policy" command, and the
Visualization News and UC/ANL TeraGrid News e-mail lists. To subscribe to these
news lists see:
http://news.teragrid.org/config/subscribe.php
The full visualization usage policies can always be found on the Web at:
http://www.uc.teragrid.org/viz/UC-ANL/policy-uc-anl.html
Site Data Policy:
-----------------
Home directories receive catastrophic backup data protection. Daily management
of data files therefore is the user's responsibility. If the data is important
to you, find a safe location for it. We *may* be in the position to retrieve a
file occasionally, but there are no guarantees. Home directory space is limited
but not directly metered. Scratch areas and parallel filesystems are not backed up and data storage for
any length of time is not guaranteed. df: `/scratch/pvfs/inca': No such file or directory
------------------------------------------
tg-login2.uc.teragrid.org policy
------------------------------------------
------------------------------------------
Available Filesystems Summary
------------------------------------------
1) Node Scratch (local scratch space on a single compute node)
Size: 60.5 GB
Location: /scratch/local, $TG_NODE_SCRATCH
Purged: after each job
Backups: none
Quota: none
Notes: 2) Cluster Scratch (scratch space shared by all compute nodes)
Size: 8745.8 GB
Location: /disks/scratchgpfs1/inca, $TG_CLUSTER_SCRATCH
Purged: as needed
Backups: none
Quota: none
Note: 3) Cluster Home (home directory space shared by all compute nodes)
Size: 475.0 GB
Location: /home/inca, $TG_CLUSTER_HOME
Purged: as needed
Backups: daily in general but normally within a week
Quota: none
Note: 4) Cluster PFS (parallel filesystem shared by all compute nodes)
Size: 8745.8 GB
Location: /disks/scratchgpfs1/inca, $TG_CLUSTER_PFS
Purged: as needed, files over 7 days first
Backups: none
Quota: none
Note: 5) Cluster GPFS (GPFS directory space shared by all compute nodes)
Size: 8745.8 GB
Location: /disks/scratchgpfs1/inca, $TG_CLUSTER_GPFS
Purged: as needed, files over 7 days first
Backups: none
Quota: none
Note: PVFS is currently unavailable. PVFS2 will be available soon.
6) Cluster PVFS (PVFS directory space shared by all compute nodes)
Size: GB
Location: /scratch/pvfs/inca, $TG_CLUSTER_PVFS
Purged: as needed, files over 7 days first
Backups: none
Quota: none
Note: tg-policy v0.2
ANL viz
Scheduling Policy:
-------------------------- General Policy:
---------------
Given the small node count at this site please submit jobs with extra
discretion. Select small node counts for long running jobs, or short
execution times for large node count jobs. Priority is given to the
realtime visualization systems between 8am-6pm. Jobs will be interrupted
if someone requires physical access to the visualization resources.
Advance notice may or may not be provided. Specific Visualization Policy:
------------------------------
Visualization work has the highest priority of all jobs in progress,
waiting or queued on the system at any time. There are two visualization
categories available: Interactive Visualization and Batch Visualization.
Interactive Visualization work using directly attached physical devices
to local nodes receive the highest priority available. Further, all
visualization jobs may and will pre-empt jobs in progress, queued, or
waiting. Interactive visualization jobs, of any type, take priority
over batch visualization. Cross Site Reservations:
------------------------
Cross site reservations are a higher priority than local reservations.
However, interactive visualization jobs using locally attached physical
devices still have a higher priority than even cross-site reservations. Changes in policy will be published here through the "tg-policy" command, and the
Visualization News and UC/ANL TeraGrid News e-mail lists. To subscribe to these
news lists see:
http://news.teragrid.org/config/subscribe.php
The full visualization usage policies can always be found on the Web at:
http://www.uc.teragrid.org/viz/UC-ANL/policy-uc-anl.html
Site Data Policy:
-----------------
Home directories receive catastrophic backup data protection. Daily management
of data files therefore is the user's responsibility. If the data is important
to you, find a safe location for it. We *may* be in the position to retrieve a
file occasionally, but there are no guarantees. Home directory space is limited
but not directly metered. Scratch areas and parallel filesystems are not backed up and data storage for
any length of time is not guaranteed. df: `/scratch/pvfs/inca': No such file or directory
------------------------------------------
tg-viz-login2.uc.teragrid.org policy
------------------------------------------
------------------------------------------
Available Filesystems Summary
------------------------------------------
1) Node Scratch (local scratch space on a single compute node)
Size: 135.6 GB
Location: /scratch/local, $TG_NODE_SCRATCH
Purged: after each job
Backups: none
Quota: none
Notes: 2) Cluster Scratch (scratch space shared by all compute nodes)
Size: 8745.8 GB
Location: /disks/scratchgpfs1/inca, $TG_CLUSTER_SCRATCH
Purged: as needed
Backups: none
Quota: none
Note: 3) Cluster Home (home directory space shared by all compute nodes)
Size: 475.3 GB
Location: /home/inca, $TG_CLUSTER_HOME
Purged: as needed
Backups: daily in general but normally within a week
Quota: none
Note: 4) Cluster PFS (parallel filesystem shared by all compute nodes)
Size: 8745.8 GB
Location: /disks/scratchgpfs1/inca, $TG_CLUSTER_PFS
Purged: as needed, files over 7 days first
Backups: none
Quota: none
Note: 5) Cluster GPFS (GPFS directory space shared by all compute nodes)
Size: 8745.8 GB
Location: /disks/scratchgpfs1/inca, $TG_CLUSTER_GPFS
Purged: as needed, files over 7 days first
Backups: none
Quota: none
Note: PVFS is currently unavailable. PVFS2 will be available soon.
6) Cluster PVFS (PVFS directory space shared by all compute nodes)
Size: GB
Location: /scratch/pvfs/inca, $TG_CLUSTER_PVFS
Purged: as needed, files over 7 days first
Backups: none
Quota: none
Note: tg-policy v0.2
Indiana
Scheduling Policy:
--------------------------
Given the small node count at this site please submit jobs with extra
discretion. Select small node counts for long running jobs, or short
execution times for large node count jobs. Priority is given to the
realtime visualization systems between 8am-6pm. Jobs will be interrupted
if someone requires physical access to the visualization resources.
Advance notice may or may not be available. Site Data Policy:
----------------- The quota for home directories is 1GB.
Daily management of data files is the user's responsibility. If the
data is important to you, find a safe location for it.
Scratch areas and parallel filesystems are not backed up. Files on
GPFS that are unmodified for longer than 90 days are purged. Files on
local scratch that are unmodified for longer than 14 days are purged.
------------------------------------------
th1.avidd.iu.edu policy
------------------------------------------
------------------------------------------
Available Filesystems Summary
------------------------------------------ 1) Node Scratch (local scratch space on a single compute node)
Size: 29.3 GB
Location: /scr, $TG_NODE_SCRATCH
Purged:
Backups: none
Quota: none
Notes: 2) Cluster Scratch (scratch space shared by all compute nodes)
Size: 812.7 GB
Location: /gpfst/inca, $TG_CLUSTER_SCRATCH
Purged:
Backups: none
Quota: none
Note: 3) Cluster Home (home directory space shared by all compute nodes)
Size: 123.1 GB
Location: /N/T/inca-ia64, $TG_CLUSTER_HOME
Purged:
Backups: daily in general but normally within a week
Quota:
Note:
4) Cluster PFS (parallel filesystem shared by all compute nodes)
Size: 812.7 GB
Location: /gpfst/inca, $TG_PFS_HOME
Purged:
Backups: none
Quota: none
Note: 5) Cluster GPFS (GPFS directory space shared by all compute nodes)
Size: 812.7 GB
Location: /gpfst/inca, $TG_GPFS_HOME
Purged:
Backups: none
Quota: none
Note:
6) Cluster PVFS (PVFS directory space shared by all compute nodes)
Size: not available at this location
Location: not available at this location
Purged: not available at this location
Backups: not available at this location
Quota: not available at this location
Note: not available at this location tg-policy v0.2
NCSA IA-64
Scheduling Policy:
--------------------------
Scheduling has a fair-share policy enabled, where every user with eligible
jobs in queue will have some of them run within a 5 day period. Only the
first 256 jobs queued for any user will be considered for scheduling
purposes. We have 3 categories of nodes (resource types) here.
fastcpu nodes have 1.5GHZ processors and access to the TG_CLUSTER_GPFS filesystem.
We have 631 of these.
fastio nodes have 1.3GHZ processors and access to both the TG_CLUSTER_GPFS and
the SAN GPFS filesystems
we have 256 of these.
himem nodes are a subset of the fastio nodes, but also have 12GB of ram per node
instead of the usual 4GB.
we have 128 of these. Jobs which do not specifiy a resource type will be assigned to the first available
set of nodes which fall within a single resource type. The scheduler will use the
fastio nodes first, if they are available.
Site Data Policy:
-----------------
Home directories receive daily backups. While we can retrieve "lost" files, we
recommend that the users make regular backups of important to the Mass Storage
system at NCSA. Users have a quota of 2GB for their home directories.
Cluster scratch is purged on a regular basis, with 5 days of grace for files.
Cluster gpfs is purged on an as needed basis. Users will be notified before
their files are purged. Neither filesystem is backed up and data storage is
therefore not guaranteed. We recommend that important data be moved to the Mass Storage system as soon as
possible. ------------------------------------------
tg-login1.ncsa.teragrid.org policy
------------------------------------------
------------------------------------------
Available Filesystems Summary
------------------------------------------ 1) Node Scratch (local scratch space on a single compute node)
Size: 61.4 GB
Location: /scr, $TG_NODE_SCRATCH
Purged: after each job
Backups: none
Quota: none
Notes: 2) Cluster Scratch (scratch space shared by all compute nodes)
Size: 1017.4 GB
Location: /scratch/inca, $TG_CLUSTER_SCRATCH
Purged: as needed
Backups: none
Quota: none
Note: nfs served, slow speed
3) Cluster Home (home directory space shared by all compute nodes)
Size: 3417.8 GB
Location: /home/ncsa/inca, $TG_CLUSTER_HOME
Purged: as needed
Backups: daily
Quota: 2 GB per user
Note: nfs served, slow speed 4) Cluster PFS (parallel filesystem shared by all compute nodes)
Size: 39090.8 GB
Location: /gpfs_scratch1/inca, $TG_CLUSTER_PFS
Purged: as needed
Backups: none
Quota: none
Note: same filesystem as TG_CLUSTER_GPFS 5) Cluster GPFS (GPFS directory space shared by all compute nodes)
Size: 39090.8 GB
Location: /gpfs_scratch1/inca, $TG_CLUSTER_GPFS
Purged: as needed
Backups: none
Quota: none
Note: GPFS NSD filesystem, fast 6) Cluster PVFS (PVFS directory space shared by all compute nodes
Size: not available at this location
Location: not available at this location
Purged: not available at this location
Backups: not available at this location
Quota: not available at this location
Note: not available at this location tg-policy v0.2
PSC BigBen
Scheduling Policy:
--------------------------
The Portable Batch Scheduler (PBS) controls all access to bigben's compute
processors, for both batch and interactive jobs. Bigben has two modes of
processing--day mode and night mode--and two queues--the batch queue and
the debug queue. Interactive jobs can be run in the batch or debug queue
and during day mode and night mode.
Bigben is in day mode Monday through Friday, 8:00 a.m. through 8:00 p.m.
System drains are run Monday through Friday targeted at 8:00 a.m. and at
8:00 p.m. to allow the switch between day and night modes. The debug queue
is turned on only during day mode. Jobs can be submitted to the debug queue
at any time but they will only run during day mode.
Jobs in the debug queue can request a maximum of 32 processors and a
maximum walltime of 30 minutes. 64 processors in total are allocated to the
debug queue. The job scheduler algorithm for the debug queue is FIFO with
anti-stuffing: a user can have only 1 debug queue job running at a time
if other users have jobs waiting in the debug queue.
The batch queue is on during day mode and night mode. During day mode the
scheduler will only start jobs in the batch queue that ask for 1024 or
fewer processors. You can submit a batch queue job that asks for more than
1024 processors at any time but it will not run during day mode. Day
mode batch queue jobs can request a maximum walltime of 5 hours. The
scheduler algorithm for the batch queue during day mode is FIFO with
anti-stuffing and backfilling
During night mode jobs in the batch queue can request as many processors as
are available on the machine and a maximum walltime of 5 hours. The night
mode scheduler favors large jobs, with short large jobs--5 minutes or less--at
the top of the queue.
You can have a maximum of 4 jobs running at a time in either the batch or
the debug queue, in day or night mode. Thus, you can have at most 8 jobs
running at one time during day mode and at most 4 running at one time during
night mode. You can submit as many jobs as you want but only 4 will
run at one time in either queue.
Site Data Policy:
-----------------
File systems are file storage spaces directly connected to a system. There are
currently two such areas available to you on bigben. Together these
two areas have over 200 Tbytes of space.
/usr/users/n/username
This is your home directory. The numeral 'n' will be replaced by an integer
and 'username' will be replaced by your userid. You can also refer to
this directory as $HOME. Your $HOME directory has a 1-Gbyte quota. $HOME is
visible to all the compute processors and all the front end processors.
$HOME is backed up daily.
$SCRATCH
This is bigben's scratch area to be used as a working space for your running
jobs. $SCRATCH is implemented using Cray's Lustre parallel file system.
There are no quotas for $SCRATCH, so you must police your own disk usage. We
will delete $SCRATCH files if we need space to keep jobs running. $SCRATCH is
visible to all the compute processors and all the front end processors.
$SCRATCH is not backed up.
------------------------------------------
tg-login9.bigben.psc.teragrid.org policy
------------------------------------------
------------------------------------------
Available Filesystems Summary
------------------------------------------
1) Node Scratch (local scratch space on a single compute node)
Size: 35100.8 GB
Location: , $TG_NODE_SCRATCH
Purged: after each job
Backups: none
Quota: none
Notes:
2) Cluster Scratch (scratch space shared by all compute nodes)
Size: 35100.8 GB
Location: , $TG_CLUSTER_SCRATCH
Purged: as needed
Backups: none
Quota: none
Note:
3) Cluster Home (home directory space shared by all compute nodes)
Size: 182.4 GB
Location: /usr/users/0/nigra, $TG_CLUSTER_HOME
Purged: as needed
Backups: daily in general but normally within a week
Quota: none
Note:
4) Cluster PFS (parallel filesystem shared by all compute nodes)
Size: not available at this location
Location: not available at this location
Purged: not available at this location
Backups: not available at this location
Quota: not available at this location
Note: not available at this location
5) Cluster GPFS (GPFS directory space shared by all compute nodes)
Size: not available at this location
Location: not available at this location
Purged: not available at this location
Backups: not available at this location
Quota: not available at this location
Note: not available at this location
6) Cluster PVFS (PVFS directory space shared by all compute nodes)
Size: 35100.8 GB
Location: , $TG_CLUSTER_PVFS
Purged: as needed
Backups: none
Quota: none
Note:
tg-policy v0.2
PSC Pople
Note: On Pople, users must load the tg-policy module prior to running tg-policy:
% module load tg-policy
% tg-policy -a
Scheduling Policy:
--------------------------
Given the small node count at this site please submit jobs with extra
discretion. Select small node counts for long running jobs, or short
execution times for large node count jobs. Priority is given to the
realtime visualization systems between 8am-6pm. Jobs will be interrupted
if someone requires physical access to the visualization resources.
Advance notice may or may not be available.
Site Data Policy:
-----------------
Home directories receive catastrophic backup data protection. Daily management
of data files therefore is the user's responsibility. If the data is important
to you, find a safe location for it. We *may* be in the position to retrieve a
file occasionally, but there are no guarantees. Home directory space is limited
but not directly metered.
Scratch areas and parallel filesystems are not backed up and data storage for
any length of time is not guaranteed.
------------------------------------------
tg-login2.pople.psc.teragrid.org policy
------------------------------------------
------------------------------------------
Available Filesystems Summary
------------------------------------------
1) Node Scratch (local scratch space on a single compute node)
Size: 25074.7 GB
Location: , $TG_NODE_SCRATCH
Purged: after each job
Backups: none
Quota: none
Notes:
2) Cluster Scratch (scratch space shared by all compute nodes)
Size: 25074.7 GB
Location: , $TG_CLUSTER_SCRATCH
Purged: as needed
Backups: none
Quota: none
Note:
3) Cluster Home (home directory space shared by all compute nodes)
Size: 25074.7 GB
Location: , $TG_CLUSTER_HOME
Purged: as needed
Backups: daily in general but normally within a week
Quota: none
Note:
4) Cluster PFS (parallel filesystem shared by all compute nodes)
Size: not available at this location
Location: not available at this location
Purged: not available at this location
Backups: not available at this location
Quota: not available at this location
Note: not available at this location
5) Cluster GPFS (GPFS directory space shared by all compute nodes)
Size: not available at this location
Location: not available at this location
Purged: not available at this location
Backups: not available at this location
Quota: not available at this location
Note: not available at this location
6) Cluster PVFS (PVFS directory space shared by all compute nodes)
Size: 25074.7 GB
Location: , $TG_CLUSTER_PVFS
Purged: as needed
Backups: none
Quota: none
Note:
tg-policy v0.2
Purdue SP
Scheduling Policy:
--------------------------
Given the small node count at this site please submit jobs with extra
discretion. Select small node counts for long running jobs, or short
execution times for large node count jobs. Priority is given to the
realtime visualization systems between 8am-6pm. Jobs will be interrupted
if someone requires physical access to the visualization resources.
Advance notice may or may not be available.
Site Data Policy:
-----------------
Home directories receive catastrophic backup data protection. Daily management
of data files therefore is the user's responsibility. If the data is important
to you, find a safe location for it. We *may* be in the position to retrieve a
file occasionally, but there are no guarantees. Home directory space is limited
but not directly metered.
Scratch areas and parallel filesystems are not backed up and data storage for
any length of time is not guaranteed.
------------------------------------------
cycle.rcac.purdue.edu policy
------------------------------------------
------------------------------------------
Available Filesystems Summary
------------------------------------------
1) Node Scratch (local scratch space on a single compute node)
Size: run "df -k /tmp" on the compute node
Location: /tmp, $TG_NODE_SCRATCH
Purged: after each job
Backups: none
Quota: none
Notes:
2) Cluster Scratch (scratch space shared by all compute nodes)
Size: 6.0 GB
Location: /scratch/local, $TG_CLUSTER_SCRATCH
Purged: as needed
Backups: none
Quota: none
Note:
3) Cluster Home (home directory space shared by all compute nodes)
Size: 66.5 GB
Location: /home/clerk/u76/inca, $TG_CLUSTER_HOME
Purged: as needed
Backups: daily in general but normally within a week
Quota: none
Note:
tg-policy v0.2
SDSC IA-64
Scheduling Policy:
--------------------------
Given the small node count at this site please submit jobs with extra
discretion. Select small node counts for long running jobs, or short
execution times for large node count jobs. Priority is given to the
realtime visualization systems between 8am-6pm. Jobs will be interrupted
if someone requires physical access to the visualization resources.
Advance notice may or may not be available.
Site Data Policy:
-----------------
Home directories receive catastrophic backup data protection. Daily management
of data files therefore is the user's responsibility. If the data is important
to you, find a safe location for it. We *may* be in the position to retrieve a
file occasionally, but there are no guarantees. Home directory space is limited
but not directly metered.
Scratch areas and parallel filesystems are not backed up and data storage for
any length of time is not guaranteed.
------------------------------------------
tg-login1.sdsc.teragrid.org policy
------------------------------------------
------------------------------------------
Available Filesystems Summary
------------------------------------------
1) Node Scratch (local scratch space on a single compute node)
Size: 68.5 GB
Location: /scratch, $TG_NODE_SCRATCH
Purged: after each job
Backups: none
Quota: none
Notes:
2) Cluster Scratch (scratch space shared by all compute nodes)
Size: 43127.0 GB
Location: /gpfs/inca, $TG_CLUSTER_SCRATCH
Purged: as needed
Backups: none
Quota: none
Note:
3) Cluster Home (home directory space shared by all compute nodes)
Size: 806.0 GB
Location: /users/inca, $TG_CLUSTER_HOME
Purged: as needed
Backups: daily in general but normally within a week
Quota: none
Note:
4) Cluster PFS (parallel filesystem shared by all compute nodes)
Size: not available at this location
Location: not available at this location
Purged: not available at this location
Backups: not available at this location
Quota: not available at this location
Note: not available at this location
5) Cluster GPFS (GPFS directory space shared by all compute nodes)
Size: 43127.0 GB
Location: /gpfs/inca
Purged: none
Backups: not backed up (treat as I/O intensive scratch space)
Quota: none
Note:
6) Cluster PVFS (PVFS directory space shared by all compute nodes)
Size: 43127.0 GB
Location: , $TG_CLUSTER_PVFS
Purged: as needed
Backups: none
Quota: none
Note:
tg-policy v0.2
TACC Lonestar
Scheduling Policy:
--------------------------
Each LSF queue is assigned a priority. When fully implemented, jobs in higher
priority queues will be scheduled before jobs in lower priority queues, given
equal (or nearly equal) Fairshare priority. Queue priority (in descending
order), as well as other information, is displayed by the
bqueues
command.
Fairshare scheduling is presently based on a single host partition, and at
present, the queue priority is actually the same across all queues within the
host partition. This will be changed to accomodate the queue priority in the
future. With the present configuration the dispatch order depends on dynamic
share priority, then order of jobs in the queue. All users are assigned a unit
fairshare; but occasionally training and benchmark accounts are assigned a
larger fairshare. Use the command
bhpart -r
to see all assigned user priorities.
Site Data Policy:
-----------------
The TACC HPC platforms have several different file systems with distinct storage
characteristics. There are predefined, user-owned directories in these file
systems for users to store their data. Of course, these file systems are shared
with other users, so they are managed by either a quota limit, a purge policy
(time-residency) limit, or a migration policy.
Home directory
The system automatically changes to a user's home directory at login and this
is the recommend location to store your source codes and build your executables.
$HOME are quota limited. Use to quota command to determine your quota.
Use $HOME to reference your home directory in scripts.
Use cd to change to $HOME.
Work directory
Store large files and perform most of your job runs from this file system.
This file system is accessible from all the nodes.
Work files are subject to a 7 day purge policy.
Use $WORK to reference your work directory in scripts.
Use cdw to change to $WORK.
Scratch or Temporary directory
Directory on each node where you can store files and perform local I/O for
the duration of a batch job. Often, in batch jobs it is more efficient to use
and store files directly on $WORK (to avoid moving files from scratch at the end
of a job). Available for the duration of a job.
Use $SCRATCH to reference a temporary directory in scripts.
San directory
The SAN directory is available on login nodes (front-ends) of Lonestar
(and Longhorn). Space on the SAN is an allocatable resource; that is,
space is not automatically allocated to a project, the Principal Investigator
must request space on this file system.
Archive directory
Store permanent files here. This file system has "archive" characteristics
(See TACC Lonestar User Guide.) The access speed is low relative to the work
directory.
Use $ARCHIVE to reference your archive directory in scripts.
Use cda to change to $ARCHIVE.
Home directories are backed up daily for protection agains catastrophic events.
Daily management of data files therefore is the user's responsibility.
Scratch areas and parallel filesystems are not backed up and data storage for
any length of time beyond the purge period is not guaranteed.
------------------------------------------
lonestar policy
------------------------------------------
------------------------------------------
Available Filesystems Summary
------------------------------------------
1) Node Scratch (local scratch space on a single compute node)
Size: 154.2 GB
Location: /work/inca, $TG_NODE_SCRATCH
Purged: after each job
Backups: none
Quota: none
Notes:
2) Cluster Scratch (scratch space shared by all compute nodes)
Size: 154.2 GB
Location: /work/inca, $TG_CLUSTER_SCRATCH
Purged: as needed
Backups: none
Quota: none
Note:
3) Cluster Home (home directory space shared by all compute nodes)
Size: 219.1 GB
Location: /home/inca, $TG_CLUSTER_HOME
Purged: as needed
Backups: daily in general but normally within a week
Quota: none
Note:
4) Cluster PFS (parallel filesystem shared by all compute nodes)
Size: not available at this location
Location: not available at this location
Purged: not available at this location
Backups: not available at this location
Quota: not available at this location
Note: not available at this location
5) Cluster GPFS (GPFS directory space shared by all compute nodes)
Size: not available at this location
Location: not available at this location
Purged: not available at this location
Backups: not available at this location
Quota: not available at this location
Note: not available at this location
6) Cluster PVFS (PVFS directory space shared by all compute nodes)
Size: 219.1 GB
Location: , $TG_CLUSTER_PVFS
Purged: as needed
Backups: none
Quota: none
Note:
tg-policy v0.2
TACC Maverick
Scheduling Policy:
--------------------------
Each LSF queue is assigned a priority. When fully implemented, jobs in higher
priority queues will be scheduled before jobs in lower priority queues, given
equal (or nearly equal) Fairshare priority. Queue priority (in descending
order), as well as other information, is displayed by the
bqueues
command.
Fairshare scheduling is presently based on a single host partition, and at
present, the queue priority is actually the same across all queues within the
host partition. This will be changed to accomodate the queue priority in the
future. With the present configuration the dispatch order depends on dynamic
share priority, then order of jobs in the queue. All users are assigned a unit
fairshare; but occasionally training and benchmark accounts are assigned a
larger fairshare. Use the command
bhpart -r
to see all assigned user priorities.
Site Data Policy:
-----------------
The TACC HPC platforms have several different file systems with distinct storage characteristics.
There are predefined, user-owned directories in these file systems for users to store their data.
Of course, these file systems are shared with other users, so they are managed by either a quota
limit, a purge policy (time-residency) limit, or a migration policy.
Home directory
The system automatically changes to a user's home directory at login and this
is the recommend location to store your source codes and build your executables.
$HOME are quota limited. Use to quota command to determine your quota.
Use $HOME to reference your home directory in scripts.
Use cd to change to $HOME.
Work directory
Store large files and perform most of your job runs from this file system.
This file system is accessible from all the nodes.
Work files are subject to a 7 day purge policy.
Use $WORK to reference your work directory in scripts.
Use cdw to change to $WORK.
Scratch or Temporary directory
Directory on each node where you can store files and perform local I/O for
the duration of a batch job. Often, in batch jobs it is more efficient to use
and store files directly on $WORK (to avoid moving files from scratch at the end
of a job). Available for the duration of a job.
Use $SCRATCH to reference a temporary directory in scripts.
San directory
The SAN directory is available on login nodes (front-ends) of Lonestar
(and Longhorn). Space on the SAN is an allocatable resource; that is,
space is not automatically allocated to a project, the Principal Investigator
must request space on this file system.
Archive directory
Store permanent files here. This file system has "archive" characteristics
(See TACC Lonestar User Guide.) The access speed is low relative to the work
directory.
Use $ARCHIVE to reference your archive directory in scripts.
Use cda to change to $ARCHIVE.
Home directories are backed up daily for protection agains catastrophic events.
Daily management of data files therefore is the user's responsibility.
Scratch areas and parallel filesystems are not backed up and data storage for
any length of time beyond the purge period is not guaranteed.
------------------------------------------
maverick policy
------------------------------------------
------------------------------------------
Available Filesystems Summary
------------------------------------------
1) Node Scratch (local scratch space on a single compute node)
Size: 8927.6 GB
Location: /work/utexas/staff/inca, $TG_NODE_SCRATCH
Purged: after each job
Backups: none
Quota: none
Notes:
2) Cluster Scratch (scratch space shared by all compute nodes)
Size: 8927.6 GB
Location: /work/utexas/staff/inca, $TG_CLUSTER_SCRATCH
Purged: as needed
Backups: none
Quota: none
Note:
3) Cluster Home (home directory space shared by all compute nodes)
Size: 200.8 GB
Location: /home/utexas/staff/inca, $TG_CLUSTER_HOME
Purged: as needed
Backups: daily in general but normally within a week
Quota: none
Note:
4) Cluster PFS (parallel filesystem shared by all compute nodes)
not available at this location
5) Cluster GPFS (GPFS directory space shared by all compute nodes)
not available at this location
6) Cluster PVFS (PVFS directory space shared by all compute nodes)
not available at this location
tg-policy v0.2
ORNL
Scheduling Policy:
--------------------------
Given the small node count at this site please submit jobs with extra
discretion. Select small node counts for long running jobs, or short
execution times for large node count jobs. This site does not provide
"official" compute resources, so jobs may be killed indiscriminately
and the batch system may be unavailable at times.
Site Data Policy:
-----------------
Home directories receive catastrophic backup data protection. Daily management
of data files therefore is the user's responsibility. If the data is important
to you, find a safe location for it. We *may* be in the position to retrieve a
file occasionally, but there are no guarantees. Home directory space is limited
but not directly metered.
Scratch areas and parallel filesystems are not backed up and data storage for
any length of time is not guaranteed.
------------------------------------------
tg-login1.ornl.teragrid.org policy
------------------------------------------
------------------------------------------
Available Filesystems Summary
------------------------------------------
1) Node Scratch (local scratch space on a single compute node)
Size: 24.9 GB
Location: /local/scratch, $TG_NODE_SCRATCH
Purged: after each job
Backups: none
Quota: none
Notes:
2) Cluster Scratch (scratch space shared by all compute nodes)
Size: 2.3 GB
Location: /scratch/inca, $TG_CLUSTER_SCRATCH
Purged: as needed
Backups: none
Quota: none
Note:
3) Cluster Home (home directory space shared by all compute nodes)
Size: 220.1 GB
Location: /users/inca, $TG_CLUSTER_HOME
Purged: as needed
Backups: daily in general but normally within a week
Quota: none
Note:
4) Cluster PFS (parallel filesystem shared by all compute nodes)
Size: not available at this location
Location: not available at this location
Purged: not available at this location
Backups: not available at this location
Quota: not available at this location
Note: not available at this location
5) Cluster GPFS (GPFS directory space shared by all compute nodes)
Size: not available at this location
Location: not available at this location
Purged: not available at this location
Backups: not available at this location
Quota: not available at this location
Note: not available at this location
6) Cluster PVFS (PVFS directory space shared by all compute nodes)
Size: 2.3 GB
Location: , $TG_CLUSTER_PVFS
Purged: as needed
Backups: none
Quota: none
Note:
tg-policy v0.2
