Storage space is available on the TeraGrid, either as companion storage
that automatically accompanies a computation allocation
or as a separate data allocation, independent of
computation allocations. Data storage allocations meet the needs of researchers
for short- and long-term
storage and for staging of data collections in databases or on disk. They are obtained
through the same proposal system that is used to request computation resources.
Each resource provider (RP) site that offers data resources for allocation through
the TeraGrid has its own policies and specifications. In addition to storage resources
available at individual RP sites, a global file system (GPFS-WAN) is mounted on
multiple platforms across the TeraGrid. The table below contains information
for each RP site, including details about networks, file space, database availability,
and recommended use associated with each storage resource and whether GPFS-WAN is available
at the resource.
(For information about storage that is associated with computation allocations,
refer to the computation resources pages of the TeraGrid Resource Catalog.)
| Resource Name |
Description & Recommended Use |
Specifications |
Media Type |
Total File Space |
Database |
Access |
|
| Dedicated (nonpurged) disk for databases and data collections |
IU Data Collections and Database Dedicated (nonpurged) Disk Space
Recommended Use
Storage of persistent data collections on disk in any standard format as well as in Oracle and MySQL databases
Status
Available for allocations and in production
|
|
Disk |
100 TB |
Oracle
MySQL |
GridFTP
from
spinning disk storage Big Red or from the IU Data Capacitor (via Lustre clients)
IU is also in the process of creating a Web portal interface for GridFTP access |
| Lustre file space (IU Data Capacitor) |
Dedicated (nonpurged) disk storage in support of data collections and data-centric computing;
Lustre is offered as an experimental service for researchers who may have a particular interest in Lustre.
Recommended Use
Storage of persistent data collections on disk in any standard format as well as in Oracle and MySQL databases
Status
Available for allocations and in production as a test service for the TeraGrid. |
|
Disk |
535 TB |
Oracle
MySQL |
via Lustre from the IU Data Capacitor
IU is also in the process of creating a Web portal interface for GridFTP access. |
| IU Archival Storage (replicated or single copy) |
Archival storage under control of the HPSS (High Performance Software System) software, stored on 500 GB tapes. The HPSS installation is a geographically distributed data storage system. Data may be copied in one location or replicated in two locations (Bloomington and Indianapolis)
Recommended Use
Storage of very large data sets, including storage of highly valuable data in two geographically distinct locations
Status
Allocable and in production
|
HPSS
500 GB tapes
52 tape drives
R/W=100 MB/sec
1- and 4-way stripes |
Tape |
2.8 PB |
NA |
directly from tape or from a front-end cache
GridFTP
HPSS Hierarchical Storage Interface (HSI)
IU recommends use of GridFTP. |
| GPFS-WAN |
TeraGrid GPFS-WAN (Global Parallel File System-Wide Area Network) is a large-scale storage
system mounted on several TeraGrid platforms. Although the system is physically located at SDSC,
it looks like it is local to the system on which it is mounted.
Recommended Use
GPFS-WAN is recommended for long- or short-term data storage of high-volume multi-site runs,
as well as for large TeraGrid-based data collections. Allocated space is available in the
Long-term Collections Area, a 150-TB partition for data collections.
Status
Available for allocations and in production
What to Choose in POPS
After choosing the amount of storage you need on the "Select the Resource Level for Your Request",
select the corresponding meeting from the "Upcoming Meetings" page:
On the Resource Request page, choose "GPFS-WAN Disk Space".
More information on GPFS-WAN
|
|
Disk |
700 TB
Total capacity
150 TB
Long-term collections
475 TG
Project Area
75 TG
Scratch
How to apply |
Not applicable |
GPFS-WAN is currently mounted on the following machines:
- IU
tg-login.iu.teragrid.org
(Big Red PPC Linux Cluster)
- NCSA
tg-login.ncsa.teragrid.org
(IA-64 Linux Cluster)
- SDSC
tg-login.sdsc.teragrid.org
(IA-64 Linux Cluster)
dslogin.sdsc.edu
(DataStar p655 and p690)
bglogin.sdsc.edu
(Blue Gene)
- UC/ANL
tg-viz-login.uc.teragrid.org
(IA-32)
tg-login.uc.teragrid.org
(IA-64 Linux Cluster)
|
NCAR |
| NCAR Mass Storage System (Single or Double Copy) |
Controls archival storage using the Mass Storage System's software. Data is stored on 200 GB tapes. NCAR MSS is located at NCAR.
Recommended Use
NCAR MSS is recommend for use in archiving data required for running jobs on frost, or for frost output that cannot be moved offsite. MSS files are limited to 12 GBs in size at this time. Storage of large data sets requires the splitting of datasets.
Status
Available for allocations and in production |
NCAR MSS
200 GB Tapes
Number of tape drives, r/w speed, striping, etc. is irrelevant because this system is for archival rather than file server purposes
|
Tape |
2 PBs
Limit per user: 5 TB-Years
|
N/A |
NCAR MSS DCS Commands (see forst man pages on msrcp command for more information, or check NCAR DCS Information Web Pages). |
| GPFS-WAN |
TeraGrid GPFS-WAN (Global Parallel File System-Wide Area Network) is a large-scale storage
system mounted on several TeraGrid platforms. Although the system is physically located at SDSC,
it looks like it is local to the system on which it is mounted.
Recommended Use
GPFS-WAN is recommended for long- or short-term data storage of high-volume multi-site runs,
as well as for large TeraGrid-based data collections. Allocated space is available in the
Long-term Collections Area, a 150-TB partition for data collections.
Status
Available for allocations and in production
What to Choose in POPS
After choosing the amount of storage you need on the "Select the Resource Level for Your Request",
select the corresponding meeting from the "Upcoming Meetings" page:
On the Resource Request page, choose "GPFS-WAN Disk Space".
More information on GPFS-WAN
|
|
Disk |
700 TB
Total capacity
150 TB
Long-term collections
475 TG
Project Area
75 TG
Scratch
How to apply |
Not applicable |
GPFS-WAN is currently mounted on the following machines:
- IU
tg-login.iu.teragrid.org
(Big Red PPC Linux Cluster)
- NCSA
tg-login.ncsa.teragrid.org
(IA-64 Linux Cluster)
- SDSC
tg-login.sdsc.teragrid.org
(IA-64 Linux Cluster)
dslogin.sdsc.edu
(DataStar p655 and p690)
bglogin.sdsc.edu
(Blue Gene)
- UC/ANL
tg-viz-login.uc.teragrid.org
(IA-32)
tg-login.uc.teragrid.org
(IA-64 Linux Cluster)
|
Resource Name
Platform |
Description & Recommended Use |
Specifications |
Media Type |
Total File Space |
Database |
Access |
|
| NCSA Data Resource Services |
Recommended Use
Status
|
|
|
|
|
|
| NCSA Database |
Recommended Use
Status Available for allocations and in production
|
|
|
|
PostgreSQL
MySQL
Oracle |
|
| NCSA Tape Storage |
Recommended Use
Status Available for allocations and in production
|
|
Tape |
|
|
|
| NCSA Project Work Space Per Host |
Recommended Use
Status Available for allocations and in production
|
|
|
|
|
|
| GPFS-WAN |
TeraGrid GPFS-WAN (Global Parallel File System-Wide Area Network) is a large-scale storage
system mounted on several TeraGrid platforms. Although the system is physically located at SDSC,
it looks like it is local to the system on which it is mounted.
Recommended Use
GPFS-WAN is recommended for long- or short-term data storage of high-volume multi-site runs,
as well as for large TeraGrid-based data collections. Allocated space is available in the
Long-term Collections Area, a 150-TB partition for data collections.
Status
Available for allocations and in production
What to Choose in POPS
After choosing the amount of storage you need on the "Select the Resource Level for Your Request",
select the corresponding meeting from the "Upcoming Meetings" page:
On the Resource Request page, choose "GPFS-WAN Disk Space".
More information on GPFS-WAN
|
|
Disk |
700 TB
Total capacity
150 TB
Long-term collections
475 TG
Project Area
75 TG
Scratch
How to apply |
Not applicable |
GPFS-WAN is currently mounted on the following machines:
- IU
tg-login.iu.teragrid.org
(Big Red PPC Linux Cluster)
- NCSA
tg-login.ncsa.teragrid.org
(IA-64 Linux Cluster)
- SDSC
tg-login.sdsc.teragrid.org
(IA-64 Linux Cluster)
dslogin.sdsc.edu
(DataStar p655 and p690)
bglogin.sdsc.edu
(Blue Gene)
- UC/ANL
tg-viz-login.uc.teragrid.org
(IA-32)
tg-login.uc.teragrid.org
(IA-64 Linux Cluster)
|
Resource Name
Platform |
Description & Recommended Use |
Specifications |
Media Type |
Total File Space |
Database |
Access |
|
| SDSC Collections Disk Space |
Development (up to 5 TB), medium 5-25 TB), or large (greater than 25 TG)
allocations are available for housing data collection in a database or on disk
Recommended Use
SDSC Collections Disk Space is recommended for allocations that are web accessible, interactive, and/or used with an application.
Status
Available for allocations and in production
|
|
Disk |
|
Oracle
MySQL |
|
| SDSC Tape Storage |
Development (up to 25 TB), medium 25-200 TB), or large (greater than 100 TG)
allocations are available for long-term archival storage independent of SDSC computation resources
Recommended Use
SDSC Tape Storage are recommended for sets of data that require high availability but do not require real-time access.
Status
Available for allocations and in production
|
HPSS |
Tape |
25 PB |
|
HSI |
| GPFS-WAN |
TeraGrid GPFS-WAN (Global Parallel File System-Wide Area Network) is a large-scale storage
system mounted on several TeraGrid platforms. Although the system is physically located at SDSC,
it looks like it is local to the system on which it is mounted.
Recommended Use
GPFS-WAN is recommended for long- or short-term data storage of high-volume multi-site runs,
as well as for large TeraGrid-based data collections. Allocated space is available in the
Long-term Collections Area, a 150-TB partition for data collections.
Status
Available for allocations and in production
What to Choose in POPS
After choosing the amount of storage you need on the "Select the Resource Level for Your Request",
select the corresponding meeting from the "Upcoming Meetings" page:
On the Resource Request page, choose "GPFS-WAN Disk Space".
More information on GPFS-WAN
|
|
Disk |
700 TB
Total capacity
150 TB
Long-term collections
475 TG
Project Area
75 TG
Scratch
How to apply |
Not applicable |
GPFS-WAN is currently mounted on the following machines:
- IU
tg-login.iu.teragrid.org
(Big Red PPC Linux Cluster)
- NCSA
tg-login.ncsa.teragrid.org
(IA-64 Linux Cluster)
- SDSC
tg-login.sdsc.teragrid.org
(IA-64 Linux Cluster)
dslogin.sdsc.edu
(DataStar p655 and p690)
bglogin.sdsc.edu
(Blue Gene)
- UC/ANL
tg-viz-login.uc.teragrid.org
(IA-32)
tg-login.uc.teragrid.org
(IA-64 Linux Cluster)
|
Resource Name
Platform |
Description & Recommended Use |
Specifications |
Media Type |
Total File Space |
Database |
Access |
UC/ANL |
| GPFS-WAN |
TeraGrid GPFS-WAN (Global Parallel File System-Wide Area Network) is a large-scale storage
system mounted on several TeraGrid platforms. Although the system is physically located at SDSC,
it looks like it is local to the system on which it is mounted.
Recommended Use
GPFS-WAN is recommended for long- or short-term data storage of high-volume multi-site runs,
as well as for large TeraGrid-based data collections. Allocated space is available in the
Long-term Collections Area, a 150-TB partition for data collections.
Status
Available for allocations and in production
What to Choose in POPS
After choosing the amount of storage you need on the "Select the Resource Level for Your Request",
select the corresponding meeting from the "Upcoming Meetings" page:
On the Resource Request page, choose "GPFS-WAN Disk Space".
More information on GPFS-WAN
|
|
Disk |
700 TB
Total capacity
150 TB
Long-term collections
475 TG
Project Area
75 TG
Scratch
How to apply |
Not applicable |
GPFS-WAN is currently mounted on the following machines:
- IU
tg-login.iu.teragrid.org
(Big Red PPC Linux Cluster)
- NCSA
tg-login.ncsa.teragrid.org
(IA-64 Linux Cluster)
- SDSC
tg-login.sdsc.teragrid.org
(IA-64 Linux Cluster)
dslogin.sdsc.edu
(DataStar p655 and p690)
bglogin.sdsc.edu
(Blue Gene)
- UC/ANL
tg-viz-login.uc.teragrid.org
(IA-32)
tg-login.uc.teragrid.org
(IA-64 Linux Cluster)
|
