GPFS-WAN
On this page
Related Links
Need Help?
Home > User Info > Data > GPFS-WAN
Description
TeraGrid GPFS-WAN (Global Parallel File System-Wide Area Network) is a 700-TB storage system mounted on several TeraGrid platforms. It has three distinct purposes, each with its own policy for access, allocation, and data preservation. The system is physically located at SDSC, but is accessible from all TeraGrid platforms on which it is mounted, appearing to the user as a local directory.
Availability
See the Data Resources page of the TeraGrid Resource Catalog for the list of resources where GPFS-WAN is currently mounted. Announcements will be made via TeraGrid User News when additional sites mount the file system in production.
Recommended Use & Policies
GPFS-WAN is recommended for long- or short-term data storage of high-volume multi-site runs, as well as for large TeraGrid-based data collections.
- Backup Policy: Users are responsible for backing up their own files; the system provides no backup for any of the partitions.
The duration of data allocations on GPFS-WAN is determined by the procedure used to create the storage area. To use GPFS-WAN, determine which purpose corresponds to your needs and follow that procedure.
| Partition Type |
Recommended Use |
Mount Point |
Size (TB) |
How to Apply |
Backup |
Purge |
Policies |
| Collections |
The Long-term Collections Area is a partition specifically for data collections that need the unique functionality of a global file system |
/gpfs-wan/
collections |
150 |
Users can request this collection space via the same peer-review process used to request CPU cycles by submitting a Data Allocation Proposal at the Partnerships Online Proposal System (POPS) |
No |
No |
|
| Projects |
The Project Area is intended as a temporary, unpurged space for multi-site analysis |
/gpfs-wan/
projects |
475 |
Any active TeraGrid allocated project which can benefit from the unique capabilities of the global file system can request space by submitting the GPFS-WAN Projects Space Request Form below. |
No |
No |
- Project directories can be created in GPFS-WAN only through the request form
- Quotas are enforced and determined based upon the request
- Duration of space availability is based on the duration of the TeraGrid project specified in the request
- Data and directories will be removed one month after the assigned TeraGrid project allocation expires
|
| Scratch |
This partition can be used for short-term data analysis before moving it to archival storage; it is available to all TeraGrid users without request; users can access the partition from any of the resources mounting GPFS-WAN, create directories, and store their data |
/gpfs-wan/
scratch |
75 |
None required. |
No |
Yes |
- Inactive files will be purged regularly in this partition as they age beyond two weeks
- Use is unlimited within 75 TB and is shared among all active users
|
These policies are subject to change; announcements will be sent via
TeraGrid User News regarding any changes to allocation, backup, or purge policy.
Technical Notes
- Access lag: The first time you access your directory on a remote site, there may be a slight lag to allow time for the UID-mapping process to complete. This does not indicate a problem with the file system. After the first access, you will see a noticeable lag only if the file system or node is heavily loaded.
- File system performance(update!!!): The peak performance of GPFS-WAN is 6.5 GB/sec at SDSC and 3 GB/sec at NCSA and Argonne. For a single node the maximum speed is 100 MB/sec at all sites. Since the bandwidth to the file system is shared among a very large number of machines at multiple sites, the performance may vary depending on the load on the file system at any given time. This is true of all parallel file systems.
- Scheduling of Jobs Using GPFS-WAN: TeraGrid systems have no automatic hold for jobs that depend on GPFS-WAN if it is unavailable. Sites that mount GPFS-WAN are expected to add support in their local batch systems to handle such events. In case of lost compute time, contact help@teragrid.org to determine if a refund is appropriate.
NCSA users only: Jobs that require GPFS-WAN should specify the gpfs-wan queue (this will prevent jobs from running during outages of the gpfs-wan file system). Use the -q gpfs-wan option on qsub for this.
GPFS-WAN Projects Space Request Form
This form is only for space requests in the Projects partition. Do not use this form for Collections or Scratch partition requests . Please use POPS for Collections partition requests. Temporary scratch storage in GPFS-WAN is available by default and does not require a request.
