TeraGrid Roaming Allocations

Home > User Support > Access > TeraGrid Roaming Allocations

Definition

TeraGrid Roaming describes the ability to access a large subset of production compute resources and other resources as part of a common allocation. This implies that all users authorized to use a Roaming allocation can run and submit HPC batch jobs on any and all such compute resources, either via traditional or grid-enabled mechanisms, and access other included resources.

Within POPS, the concept is named TeraGrid Wide Roaming Access, but it is most commonly called TeraGrid Roaming, or simply Roaming.

See the TeraGrid Resources Catalog to see which resources are currently available for TeraGrid Roaming.

Recommended Use Guidelines

TeraGrid Roaming access is appropriate for a number of usage scenarios, including taking advantage of multiple resources via grid-based software and capabilities. Roaming can also be used to permit a research group to port codes, conduct test runs, and evaluate a variety of architectures. A project may intend or need to conduct multi-site, multi-resource runs. A project may also have portable code that can be run on a variety of platforms, and Roaming access provides a mechanism for advantageously submitting jobs to resources with the shortest queues. Other usage scenarios that can leverage this unique Roaming capability of TeraGrid are also encouraged.

Requesting TeraGrid Roaming Access

Research allocations requests have the option of requesting allocations on specific resources, on Roaming resources, or both. Such requests must include a resource justification for TeraGrid Roaming requests. In most ways, a successful Roaming justification is similar to a justification for time on a specific resource. A TeraGrid Roaming request should also describe (a) why Roaming is an appropriate mechanism for delivering SUs to the project, and (b) any assumptions made to derive the total number of compute SUs requested.

For example, a Roaming request may be based on performance and scaling data on a specific resource, so the request should note any SU conversion factors applied (see Roaming SUs and Usage below).

Requests for Startup allocations can also include up to 50,000 SUs of TeraGrid Roaming for the purpose of evaluating a wide variety of architectures, or other reasons. For Research Allocations, there is no limit to the number of SUs in the allocation that can be requested or allocated as roaming SUs.

Adding Resources to a Roaming Allocation

A Roaming Allocation encompasses the production compute resources, and possibly other types of resources, that are part of the Roaming set at the time an allocation is created. New resources that come online are not automatically made part of an existing roaming allocation.

TeraGrid compute resources may come online and become part of Roaming at any time after a given allocation is created. A PI can request that any or all such resources be added to his/her Roaming allocation. Requests can be submitted via the TeraGrid User Portal or via email to help@teragrid.org.

To see current resources included in TeraGrid-Wide Roaming Access, go to the TeraGrid Resources Catalog.

Exception for Specific Allocations

If an allocated research project receives both resource-specific allocations and a Roaming allocation, the Roaming allocation in this case is defined to encompass all production compute resources active at the time the allocation is created except those on which the project has a specific allocation. In other words, Roaming in this situation is available everywhere else.

Storage and Non-Compute Resources

By default, storage and other non-compute resources are not included as part of TeraGrid Roaming. However, RPs can opt to have any TeraGrid resource added to TeraGrid Roaming. In practice, the accounting system cannot currently distinguish between storage and compute resources within TeraGrid Roaming. If an RP makes a non-compute resource part of Roaming, any charges to be made against the allocation resulting from usage of that resource will be described in the TeraGrid Resource Catalog.

Roaming SUs and Usage

Roaming allocations are awarded in TeraGrid Service Units (SUs), which are subject to local site and resource interpretation. TeraGrid does provide the following general definition:

One TeraGrid SU is defined as one processor-hour of wallclock time on a TeraGrid Phase-1 DTF cluster. For usage on a resource that is charged to a Roaming allocation, a normalization factor is applied. The normalization factor is based on the method historically used to calculate Normalized Units (NUs). These are SU equivalents on a Cray X-MP system, and are derived from the HPL benchmark performance of that resource.

These definitions can be even more resource-specific. Site-specific policies and job queues may apply locally defined charge factors that can affect the SU calculation. For example, a premium is charged for use of high-priority queues.

TeraGrid provides a basis for the calculation. The below formula is used in the TeraGrid SU Conversion Calculator, which is useful for converting SUs from one TeraGrid resource to another.

One Phase-1 DTF Service Unit is equal to 21.576 Normalization Units. The TeraGrid SU conversion factor for a resource is calculated by taking its NU conversion factor and dividing it by 21.576. The standard formula for calculating the NU conversion factor of a resource is:

(Rmax * 1000 / 191) / P

where Rmax is the Rmax result of the resource on the HPL benchmark in Gflops and P is the number of processors used in the benchmark. In the absence of an HPL benchmark run, a conversion factor can be agreed upon, based on that of an architecturally similar platform and scaled according to processor performance differences.

Conversion to Roaming SUs is handled by the TeraGrid central accounting system, and RPs are only required to report usage in local SUs for all allocations.

Defining an SU charge for specialized compute resources (such as visualization hardware) or non-compute resources (such as storage) is possible, but there is no TeraGrid-wide policy for doing so.