Globus User Guide
Globus is software-as-a-service for file transfer and sharing. Designed specifically for researchers, Globus provides fast, reliable, and secure file transfer among XSEDE resources or between an XSEDE resource and another machine (such as a campus cluster, lab server, or personal computer). Globus is core campus bridging technology that enables researchers to scale their computational research from the desktop, across campus, and to national cyberinfrastructure. Beyond file transfer, Globus allows researchers to securely share data with collaborators directly from existing systems, without investing in additional campus or cloud storage just for the purposes of sharing.
Globus is used for moving data between any two resources, whether it is a small number of very large (even terabyte-sized) files or a very large number of small files. Note: A resource is represented in Globus as an endpoint, identified by a unique name (e.g., the endpoint for TACC's Stampede is "
xsede#stampede", and NICS's Kraken is "
Globus can be used to move data:
- Between two XSEDE resources
- Between an XSEDE resource and your personal laptop or desktop
- Between an XSEDE resource and another HPC system on your campus
- Between any two non-XSEDE systems
Globus also provides an easy way to share data with collaborators directly from an existing storage system. There is no need to move data to a different system to public cloud service just for the purposes of sharing.
In order to use Globus you must have a Globus account, that you can create by going to https://www.globus.org/signup. After signing up you will receive an e-mail request to confirm your e-mail address; click on the link to confirm and you will then be able to access the file transfer and sharing service.
Note: You may also, optionally, link your Globus account to your XSEDE User Portal (XUP) identity – this will enable you to access the Globus service using your XSEDE user name and password.
Globus can be used to move data between any two XSEDE resources. All XSEDE resources are already configured as Globus endpoints.
It's easy to set up a Globus endpoint on a user's machine and upload or download files between this machine and an XSEDE resource. Globus makes it possible to easily transfer files to and from any machine – even if behind a firewall or NAT without any administrative privileges – using Globus Connect Personal.
Any multi-user HPC or shared storage system can be configured as a Globus endpoint using Globus Connect Server. On most servers, installing Globus Connect Server requires just a few commands. Once the software is installed, any user with a local account on that system can move and share files between it and an XSEDE resource. Information on Globus Connect Server is available on the Globus support web site.
Globus can also be used to move data between any two systems such as a campus cluster, a non-XSEDE computing facility, or a machine that captures data from an instrument – even if the machine is behind a firewall or NAT. You can easily turn any system into a Globus endpoint using Globus Connect.
You can share files from an XSEDE resource or any other Globus endpoint that has been enabled for sharing. Your collaborators need only a Globus account to access files that you have shared with them. Globus ensures that access controls on the local system are enforced while allowing other Globus users to easily access and download the data.
A Globus endpoint can be enabled for sharing by subscribing to Globus Plus (for personal computers) or to a Globus Provider plan (for multi-user servers). As of May 2014 only the XSEDE resources managed by the San Diego Supercomputing Center are enabled for sharing. If you would like to enable your resource for sharing please contact the Globus team about a subscription.
Globus provides a command line interface (CLI) that may be accessed using any standard SSH terminal client. Prior to running shell commands, you must upload your public SSH key to your Globus account. See the instructions in this post for more details.
Once your key is uploaded, to enter a secure Globus shell type:
% ssh email@example.com
You will see the Globus command prompt:
Welcome to globus.org, . Type "help" for help. $ _
Commonly used commands include:
- Display recently used endpoints with
- Start a transfer with
- Display the transfer tasks status with
A complete list of Globus commands, along with with detailed descriptions, is available on the CLI support forum.
Users can turn off email notifications by running the following command:
login1$ profile -n off
This command will also disable alert emails, such as a notification that your credentials have expired, requiring you to keep a slightly closer watch on long-running transfer requests.
In addition to the Web and command line interfaces described above, Globus provides programmatic access the file transfer via a REST-style API. The API is a great way to integrate Globus services into your custom portal or science gateway. The API provides Python and Java client libraries.
The Globus service is quite powerful and includes many features not mentioned above (e.g., group management, monitoring, endpoint management, etc.) You can learn more using the following resources:
- Getting started with Globus file transfer
- Frequently Asked Questions
- Sharing data using Globus
- Getting started with the Globus command line interface (CLI)
Last update: May 27, 2016