TeraGrid is composed of multiple partner institutions, each of which contributes one or more hardware resources to the grid. Resources include computational and visualization machines, data storage, data collections, and instruments. This page summarizes TeraGrid hardware resources at each of the partner sites.
For information on individual resources click the machine name in the table below to go to that provider's local Web site.
Most resources are available for TeraGrid Wide Roaming Allocations unless stated otherwise.
Resource Name
Platform |
Description |
Specifications |
| IU |
IU Big Red
IBM e1350
|
Big Red has 768 IBM JS21 compute nodes, each with two dual-core 2.5 GHz PowerPC 970MP CPUs, 8 GB memory, 72 GB of local scratch disk, and a PCI-X Myrinet-2000 interconnect for high-bandwidth low-latency MPI applications. It has access to 266 TB of local GPFS scratch space, the TeraGrid-wide GPFS-WAN file system, and to the 535 TB Lustre file system provided by the Data Capacitor. NOTE: 6.5 TFlops is available for TeraGrid usage.
Recommended Use
Big Red is a distributed shared-memory cluster, intended to run parallel as well as serial applications.
Status
Big Red is available for allocations and in production. |
Machine Type
MPP
Operating System
SuSE Linux Enterprise Server 9
Teraflops
30.7
Disk Size
266 TB |
IU Quarry
IBM IBM HS21 Bladeserver
|
Quarry is a distributed shared-memory cluster, consisting of 112 IBM HS21 Blade servers with gigabit ethernet interconnects. Each HS21 blade contains two Intel Xeon 5335 quad-core processors, 8GB of memory, and a 36GB SAS disk for local scratch space. Global scratch space is provided by 266TB GPFS and 535TB Lustre file systems.
The Quarry cluster runs the RedHat Enterprise Linux Server operating system, and uses TORQUE (PBS) and Moab for job management.
Recommended Use
This machine targets Scientific Gateways allocations. The Quarry resource is restricted to members of approved XRAC grants which have a web services component.
Status
Pre-production, available for allocations
Production Date
TBD |
Machine Type
Cluster
Operating System
RedHat Enterprise Linux Server
Teraflops
7
Disk Size
266 TB |
| LONI |
LONI Queen Bee
Dell Intel 64 Linux Cluster
|
Queen Bee is a 50.7 TFlops Peak Performance 668 node Dell PowerEdge 1950 cluster running the Red Hat Enterprise Linux 4 operating system. Each node contains two Quad Core Intel Xeon 2.33GHz 64-bit processors and 8 GB of memory. The cluster is interconnected with 10 Gb/sec Infniband and has total 192TB (raw) of storage in shared Lustre filesystems.
Recommended Use
Queen Bee is primarily intended for parallel applications scalable up to 5344 processing cores.
Note: Starting with April 1 2008 allocations, LONI Queen Bee system will be co-allocated with the NCSA Abe system.
Status
In production and accepting allocations.
|
Machine Type
Cluster
Operating System
Red Hat Enterprise Linux 4 (Linux 2.6.9-55)
Teraflops
50.7
Disk Size
192 TB |
| NCAR |
NCAR Frost
IBM Blue Gene/L
|
Frost is a single-rack, Blue Gene/L system with 1024 compute nodes. There is one I/O node for every 32 compute nodes (pset size of 32) for a total of 32 I/O nodes in the rack. Each compute node and I/O node is a dual-core chip, containing two 700MHz PowerPC-440 CPUs, 512MB of memory, and two floating-point units (FPUs) per core. Thus frost has a total of 2048 processors capable of sustaining a peak performance of 5.734 trillion floating-point operations per second (TFLOPs). By default, the compute nodes run in coprocessor mode (one processor handles computation and the other handles communication), but virtual node mode is also available, where both processors share the computation and communication load.
Recommended Use
The Frost Blue Gene/L system is a highly scalable platform for developing, testing and running parallel MPI applications up to 2048 processors, and providing efficient computing for smaller job sizes.
Status
Frost is available for requesting development allocations (DAC) starting July, 2007. Requests will be accepted starting Sept. 17, 2007, for medium and large allocations (MRAC/LRAC) beginning Jan. 1, 2008.
**** This resource is NOT included in TG WIDE ROAMING allocations ***** |
Machine Type
MPP
Operating System
SuSE Linux Enterprise Server 9
Teraflops
5.734
Disk Size
6 TB |
| NCSA |
NCSA Abe
Dell Intel 64 Linux Cluster
|
This Dell blade system has 1,200 PowerEdge 1955 dual socket, quad core compute blades, an InfiniBand interconnect and 100 TB of storage in a Lustre filesystem.
Recommended Use
The NCSA Intel 64 cluster (Abe) is intended for highly scalable parallel applications.
Note: Starting with April 1 2008 allocations, Abe will be co-allocated with the LONI Queen Bee system.
Status
Abe is in production as of July 9, 2007.
**** This resource is NOT included in TG WIDE ROAMING allocations ***** |
Machine Type
Cluster
Operating System
Red Hat Enterprise Linux 4 (Linux 2.6.9)
Teraflops
89.47
Disk Size
100 TB |
NCSA Cobalt
SGI Altix
|
The NCSA SGI Altix consists of two systems each with 512 Intel Itanium 2 processors running the Linux operating system. The systems have 1 and 3 TB of memory respectively. It uses the CXFS shared parallel filesystem from SGI.
Recommended Use
The NCSA SGI Altix (cobalt) is intended primarily for running large shared-memory applications.
Status
In production and accepting allocations. |
Machine Type
SMP
Operating System
SGI ProPack 4.0.3
Teraflops
6.55
Disk Size
100 TB |
NCSA Mercury
Intel IA-64 Cluster
|
NCSA 's IA-64 TeraGrid Linux Cluster consists of 887 IBM nodes: 256 nodes with dual 1.3 GHz Intel Itanium 2 processors (half with 4 GB of memory per node, and the other half with 12 GB of memory per node), and 631 nodes with dual 1.5 GHz Intel Itanium 2 processors (4 GB of memory per node). The cluster is running SuSE Linux and is using Myricom 's Myrinet cluster interconnect network, and the GPFS parallel filesystem.
Recommended Use
The NCSA IA-64 Linux Cluster (mercury) is primarily intended to run applications of moderate to high levels of parallelism, particularly those needing a 64-bit environment and codes that perform well in a distributed cluster environment.
Note: mercury is co-allocated with the SDSC and ANL IA-64 Clusters.
Status
In production and accepting allocation requests |
Machine Type
Cluster
Operating System
Linux 2.4.21-SMP
Teraflops
10.23
Disk Size
60 TB |
NCSA Tungsten
Dell Xeon IA-32 Cluster
|
This cluster is based on Dell PowerEdge 1750 servers, each with two Intel IA-32 Xeon 3.2 GHz processors and 3 GB of memory. It is running Red Hat Linux and Myricom 's Myrinet cluster interconnect network, and the CFS Lustre shared parallel filesystem.
Recommended Use
The NCSA Xeon Linux Cluster (tungsten) is primarily intended to run applications of moderate to high levels of parallelism, particularly those needing a 32-bit environment and codes that perform well in a distributed cluster environment.
Status
In production and accepting allocation requests |
Machine Type
Cluster
Operating System
RedHat Linux 9
Teraflops
16.38
Disk Size
109 TB |
| NICS |
NICS Kraken
Cray XT4
|
Kraken consists of 3,744 nodes interconnected with SeaStar, a 3D Torus.
Recommended Use
Kraken is intended for highly scalable parallel applications.
Status
Pre-production, available for allocations
Production Date
TBD
**** This resource is NOT included in TG WIDE ROAMING allocations ***** |
Machine Type
MPP
Operating System
Compute Node Linux
Teraflops
170
Disk Size
350 TB |
| ORNL |
ORNL NSTG Cluster
IBM IA-32 Cluster
|
The ORNL NSTG cluster has 28 nodes, 16 of which are dedicated to running compute jobs. Each compute node has two 3.06 GHz Intel Pentium4 Xeon CPUs, 2.5 GB memory, and 26 GB of local scratch. 800 GB of shared scratch is provided across the private gigabit interconnect. Four additional nodes are dedicated to running GridFTP servers, and each is configured with 4 GB of memory.
Recommended Use
Status
In production and accepting allocation requests |
Machine Type
Cluster
Operating System
SuSE Linux 10.2
Teraflops
0.34
Disk Size
2.14 TB |
| PSC |
PSC BigBen
Cray XT3
|
BigBen is a Cray XT3 MPP system with 2068 2.6-GHz dual-core AMD Opteron compute nodes linked by a custom-designed interconnect. Twenty-two dedicated I/O processors are also connected to this network. Each compute node has 2 Gbytes of memory shared by its two cores, and runs the Catamount operating system. The front end processors run SuSE Linux.
Recommended Use
BigBen is primarily intended to run applications with very high levels of parallelism or concurrency (512 - 4096 processes).
Status
In production and accepting allocation requests |
Machine Type
MPP
Operating System
SuSE Linux [Frontend] Catamount [Compute]
Teraflops
21.5
Disk Size
100 TB |
PSC Pople
SGI Altix 4700
|
Pople is a SGI Altix 4700 comprising 192 blades with 8 GB of memory and 2 sockets on each blade. Each socket is a 1.66GHz dual-core Intel Itanium 2 processor (Montvale). Pople has a total of 384 sockets, 768 cores and 1.5 TB (2GB per core) of RAM.
The blades are linked with a NUMAlink interconnect.
Recommended Use
Pople is intended for applications utilizing shared memory and hybrid architectures.
Status: Pople is accepting requests for allocations that will start July 1, 2008 |
Machine Type
SMP
Operating System
SuSE Linux
Teraflops
5.0
Disk Size
150 TB |
PSC Rachel
HP Alpha EV7
|
Rachel is a loosely-coupled pair of SMP machines. Each system has 64 1.15 GHz EV7 processors with 256 Gbytes of shared memory. Logins are to a front end node with 2 EV67 processors. Both the front end node and the SMP machines run the Tru64 Unix operating system.
Recommended Use
Rachel is primarily intended to run applications with low to moderate parallelism that require large memory bandwidth and/or large shared memory. The queuing policies favor jobs that request more shared memory.
Status
- In production but no longer accepting allocation requests
- Rachel will no longer be available after 7/1/2008
|
Machine Type
SMP
Operating System
Tru64 Unix
Teraflops
0.31
Disk Size
6.00 TB |
| Purdue |
Purdue Brutus
SGI 450
|
These resources consist of an SGI 450 (brutus.rcac.purdue.edu) with two RC100 FPGA blades, totaling 4 available FPGAs. Also available is a Sun Fire X2200 M2 (portia.rcac.purdue.edu) which serves both as a place & route node for preparing FPGA code for use on Brutus and as an entry point for GSI-SSH and job submission to Brutus by TeraGrid users.
NOTE: All references to CPU below should be interpreted as referring to be FPGAs.
Recommended Use
This resource should only be used for FPGA accelerated applications.
Status
In production and accepting allocation requests. |
Machine Type
Cluster
Operating System
SUSE Linux
Teraflops
0.042
Disk Size
22 TB |
Purdue Steele
Dell 1950 Cluster
|
The Steele cluster consists of 812 dual quad-core Dell 1950 compute nodes, running Red Hat Enterprise Linux, version 4. Each node thus has 8 64-bit Intel 2.33 GHz E5410 CPUs and either 16 GB or 32 GB of RAM. They are interconnected with either Gigabit Ethernet or InfiniBand. The machine offers access to the RCAC scratch space. Steele users also may also access a 1.3 PB DXUL archive system. Steele 's peak performance is rated at 59.94 TFLOPS.
Recommended Use
Steele is well suited for a wide range of both serial and parallel jobs.
Status
In production and accepting allocation requests |
Machine Type
Cluster
Operating System
RedHat Enterprise 4
Teraflops
59.94
Disk Size
130 TB |
Purdue TeraDRE
TeraDRE
|
The Purdue TeraDRE is a high-throughput visualization resource built on the Purdue Condor Pools. A 48-node subcluster featuring Nvidia GeForce 6600 GT GPUs are available for GPU-accelerated programs and hardware-accelerated rendering using
Gelato.
Recommended Use
The TeraDRE allows TeraGrid users to render graphics with a number of rendering packages: Maya, POV-ray, and Blender, among others.
Status
In production and accepting allocation requests.
TeraDRE is a TeraGrid visualization resource. |
Machine Type
Cluster
Operating System
Linux Debian Etch, RHEL4
Teraflops
60
Disk Size
170 TB |
Purdue
Condor Pool
|
The Purdue Condor pools consist of over 14000 CPUs of computation: 8000 LINUX/X86_64 CPUs, 400 LINUX/INTEL (ia32) CPUs, and 5000 WINNT51/INTEL CPUs, as well as a small number of Itanium Linux, Solaris and MacOS X machines. Memory on compute nodes range from 512 MB to 32 GB, and most CPUs run at 3 GHz or better. With a total of over 60 TFLOPS available, the Purdue Condor pools can provide large numbers of cycles in a short amount of time. All shared areas and software packages available on Lear are available on Condor.
Recommended Use
Condor is designed for high-throughput computing, and is excellent for parameter sweeps, Monte Carlo simulation, or most any serial application. Also, some classes of parallel jobs (master-worker) may be run in Condor.
Status
In production and accepting allocation requests |
Machine Type
Cluster
Operating System
Linux Debian Etch, RHEL4
Teraflops
60
Disk Size
170 TB |
| SDSC |
SDSC BlueGene
IBM BlueGene/L
|
The IBM Blue Gene system at SDSC is housed in three racks with 3072 compute nodes and 384 I/O nodes. This maximum ratio of I/O to compute nodes was chosen to support data-intensive computing. Each node consists of two PowerPC processors that run at 700 MHz and share 512 MB of memory, giving an aggregate peak speed of 17.2 teraflops and a total memory of 1.5 TB. Blue Gene has achieved 13.83 teraflops on the widely quoted Linpack benchmark.
Recommended Use
Blue Gene is primarily intended to run codes that scale very well, have only fine-grained parallelism, and do not use much memory per node.
Status
In production but no longer accepting allocation requests |
Machine Type
MPP
Operating System
SuSE Linux 9 [Frontend] BLRTS [Compute]
Teraflops
17.2
Disk Size
19.50 TB |
SDSC DataStar
IBM pSeries 655
|
DataStar has 272 8-way P655+ compute nodes -- 176 nodes with 1.5-GHz Power4+ CPUs and 16 GB of memory, and 96 with 1.7 GHz Power4+ CPUs and 32 GB of memory. The nodes are connected by an IBM high-speed Federation switch and have access to 130 TB of GPFS.
Recommended Use
The DataStar p655 partition is primarily intended to run applications of very high levels of parallelism or concurrency, especially those with high parallel I/O requirements. The queuing policies favor jobs with a higher processor count.
Status
In production but no longer accepting allocation requests |
Machine Type
MPP
Operating System
AIX 5.2L
Teraflops
14.30
Disk Size
115.00 TB |
SDSC DataStar
IBM pSeries 690
|
DataStar has 6 32-way P690 compute nodes with 1.7 GHz Power4+ processors. Five nodes have 128 GB of shared memory, and one node has 256 GB of memory. All nodes are connected via DataStar 's Federation switch and have access to 130 TB of GPFS.
Recommended Use
The p690s are primarily intended to run applications that require large amounts of shared memory.
Status
In production but no longer accepting allocation requests |
Machine Type
MPP
Operating System
AIX 5.2L
Teraflops
1.30
Disk Size
115.00 TB |
SDSC
IBM IA-64 Cluster
|
SDSC 's TeraGrid cluster is composed of 262 IBM Itaninum2 nodes with two processors per node. Each node is built with SuSE Linux and interconnected with Myricom 's Myrinet. The system has a peak performance of 3.1 Teraflops, a total memory of 1 TB, and a total of 50 TB GPFS disk through the SAN network. Jobs are scheduled and run by the Catalina-scheduler and PBS-batch system.
Recommended Use
Status
In production and accepting allocation requests |
Machine Type
Cluster
Operating System
Linux 2.4 (SuSE 8.0)
Teraflops
3.10
Disk Size
34 TB |
| TACC |
TACC Lonestar
Dell PowerEdge 1955
|
The Lonestar Dell PowerEdge Linux Cluster is configured with 5,840 compute-node cores, 11.6 TB of total memory and 106TB of local disk space. The peak performance rated is 62 TFLOPS. The system supports a 70TB globally accessible, Lustre parallel file system. Nodes are interconnected with InfiniBand technology in a fat-tree topology with a 1GB/sec point-to-point bandwidth. Also, a 2.8 petabyte archive system and a 5TB SAN are available through the login/development nodes.
Recommended Use
Lonestar is intended primarily for parallel applications scalable up to 4096 processing cores. Normal batch queues enable users to run up to 24-hour simulations that utilize up to 512 cores. Simulations requiring longer run times and/or more cores are accommodated via a special queue after approval from TACC technical staff. Serial and development queues are available to users for code development, conversion of serial applications to parallel, and single-cpu performance analysis.
Status
In production and accepting new allocations. |
Machine Type
MPP
Operating System
Linux (Centos) 2.6.12
Teraflops
62.16
Disk Size
106.5 TB |
TACC Maverick
Sun UltraSPARC IV
|
Maverick is a unique configuration that has resulted from a joint project between Sun Microsystems and the Texas Advanced Computing Center to provide terascale remote visualization capabilities to UT Austin researchers and the national community through the TeraGrid. This remote visualization machine leverages the vast capabilities of Sun’s E25K enterprise server with the addition of commodity graphics now providing unprecedented graphics performance.
Recommended Use
The TACC Visualization System Maverick is intended for serial and parallel, visualization applications that take advantage of a large shared memory and multiple compute processors (up to 128) and graphics processors (32). The TACC developed, web-based EnVision software tool enables user to remotely execute serial and parallel visualization applications and easily manipulate terascale data sets. During prime-time hours Maverick is dedicated to interactive use with a batch queue available for computational use during non prime-time hours.
Status
In production and accepting new allocations
Maverick is a TeraGrid visualization resource. |
Machine Type
SMP
Operating System
Solaris 9
Teraflops
0.27
Disk Size
0.56 TB |
TACC Ranger
Sun Constellation System
|
The Ranger Sun Constellation Cluster is configured with 3,936 four-socket, quad-core AMD Opteron nodes (62,976 compute cores) and 125 TB of distributed memory. With each core clocked at 2.0 GHz and capable of four flops/clock cycle, Ranger provides the user community access to a resource with a theoretical peak performance of 504 TFLOPS. Multiple shared file systems (HOME, WORK, and PROJECTS) are configured from 1.7 PB of raw storage. All file systems are managed via the Lustre Parallel File System. Nodes are interconnected with InfiniBand technology with two non-blocking Sun Magnum switches acting as the core of the fabric.
Recommended Use
Ranger is intended for users with codes scalable to thousands of cores (1024 and above). A batch queue is available to assist users develop, test, and scale codes up to 1024 compute cores. Four separate login nodes will provide interactive connectivity to the system for compiling and interfacing with the batch queuing system.
Status
In production, accepting allocation requests
Production Date
Mon., Feb. 4, 2008
**** This resource is NOT included in TG WIDE ROAMING allocations ***** |
Machine Type
MPP
Operating System
Linux (CentOS)
Teraflops
504
Disk Size
1730 TB |
| UC/ANL |
UC/ANL
IA-32 Visualization Cluster
|
UC/ANL's IA-32 TeraGrid Linux Visualization Cluster consists of 96 nodes with dual Intel Xeon processors, with 4 GB of memory and nVidia GeFORCE 6600GT AGP graphics card per node. The cluster is running SuSE Linux and is using Myricom's Myrinet cluster interconnect network. There is a 16 TB local high-performance GPFS, and access to the TeraGrid-wide GPFS-WAN file-system.
The IA-32 Visualization Cluster is a TeraGrid visualization resource. |
Machine Type
Cluster
Operating System
SuSE Enterprise Server 8
Teraflops
0.61
Disk Size
4.00 TB |
UC/ANL
Intel IA-64 Cluster
|
UC/ANL 's IA-64 TeraGrid Linux Cluster consists of 62 nodes with dual Intel Itanium 2 processors, with 4 GB of memory per node. The cluster is running SuSE Linux and is using Myricom 's Myrinet cluster interconnect network. There is
a 16 TB local high-performance GPFS, and access to the TeraGrid-wide GPFS-WAN file-system. |
Machine Type
Cluster
Operating System
SuSE Enterprise Server 8
Teraflops
0.61
Disk Size
4.00 TB |
