Gateways for PIs
The Science Gateways for PIs section brings together information required by principal investigators (PIs) to plan, organize, and oversee an XSEDE Science Gateway. Content on this page focuses on decision-making and management.
Are You Eligible?
Eligibility of a PI for sponsoring an XSEDE Science Gateway is the same as for an XSEDE allocation; in brief, you must be a U.S. researcher, or collaborating with a U.S. researcher. For details on eligibility guidelines, see NSF Resource Allocations Policies: Eligibility.For more eligibility details please view the Allocations section.
Is a Science Gateway Right for Your Community?
Before taking the first steps to start building a gateway, PIs need to assess if a gateway will add value for their research communities. A number of factors may define the usefulness of a gateway or portal. For example, you may intend to grow your user community significantly, or you may intend to develop complex workflows for only a few advanced scientists, but provide them with tremendous capabilities. The expected size of your user community and the anticipated scientific impact should justify the investment in building a gateway.
Computational Expertise and Culture of Your Community
Lack of expertise or infrequency of use of high-performance computing (HPC) resources can present obstacles to researchers trying to accomplish their computational goals in a timely manner. Your research audience may not be familiar with the Unix command line or with the complexity of optimization parameters of analytical or domain-specific computational software. Field or bench researchers may spend months obtaining experimental or observational data and only a few weeks during a grant cycle analyzing their data on computational resources. Others may be accustomed to working only on local clusters.
For any of these potential users, a portal may facilitate their computational objectives.
Data and Storage
A portal can meet the need to stage, store, and share large quantities of data or to bring data together from diverse storage systems.
Goals or Problems to Be Solved
Develop a clear idea of what, specifically, would constitute success in your gateway. Many of the qualitative criteria that you define to help you decide whether a gateway is the right solution for your community's needs may turn into quantitative success metrics later. This would also be something you describe in an allocation request for XSEDE resources. To learn more about success metrics, see the section below.
Role of XSEDE in Your Gateway
When evaluating whether to develop or extend a portal to connect to XSEDE resources, consider the advantages of using high-end resources, since extra labor is required for XSEDE integration. What scientific questions can be answered through the use of XSEDE resources that couldn't be answered otherwise? How do XSEDE capabilities add value to the scientific community? What specific bottlenecks or problems will be addressed through the use of XSEDE resources? Once you've decided to create a portal, answers to these questions will help your development team to design an effective gateway that meets the specific needs of the scientists who will use it.
How to Measure and Show Success of Your Gateway?
Gateway reporting requirements are no different than those for other users. Justification of the resources used and papers published because of the use of the resources are key factors in a successful proposal. This type of information may be more challenging for a gateway to collect because of the disparate nature of the user community. Nevertheless, science success due to the use of the gateway are important, both for XSEDE resource requests and likely for independent gateway funding requests as well.
The list below includes potential criteria for success. Specific, measurable events may be presented as numeric counts or as increases in counts over previous intervals:
- Publications generated by the community using the portal
- Hits or visits to portal interface
- Data downloads
- Simulations run
- Classes, courses, or students using a gateway
- Consumption of CPU hours on XSEDE resources
- Growth of the community (number of new users)
- Distinct institutions accessing the gateway
- Advancement of state of the art capabilities for scientists
How Much Effort Is Involved?
The success of your gateway depends on the involvement of very capable developers. The cyberinfrastructure underlying science gateways is still quite complex, and hiding the complexity from researchers is not a trivial undertaking. If possible, hire and involve the gateway development team early to help make critical decisions that will define the technology of your portal.
Attendees at the Build a Gateway tutorial at TeraGrid '07 built a simple gateway in about two hours using SimpleGrid. This was an extremely basic exercise that allowed attendees to see the different building blocks and moving parts of a gateway.
Very full-featured and evolving gateways might occupy 10 developers. However, for many projects, one or two developers are sufficient. Developers may be more engaged early on and involved more in maintenance tasks later. However, we have seen that as gateways are used, they evolve; there are nearly always new requests from users that require ongoing development.
As with any user-facing application, a wide range of technical skills is required to create a professional-looking and fully functional gateway. The specific skill and knowledge set of your team will depend on the scope of the gateway and the technologies that are chosen. Consider hiring developers with skills in a wide range of web applications, development environments, and languages (depending on the framework your portal uses)
- Gathering and translating user requirements
- Usability and interface design
- Web server administration
- SQL (basic query skills) and database administration
- High performance computing, data storage, and grid technologies (Globus)
- Quality assurance/quality control
- Web application security
- Graphics skills
What Kind of Science Gateway Should You Create?
Interface and Backend Services
You may choose to build a Web-based user interface or an application interface that is installed directly on the end-users' workstation. The gateway may connect to only XSEDE services on the back end, or it may serve as a bridge to both XSEDE services and other community grids.
Because the requirements of research communities differ, there is no single plug-and-play gateway, complete with all the modules and features that your community may need. Your experience as a researcher will provide the best guide for your initial requirements. Peruse the Gateway List to view the features that other gateways provide for their communities.
Gateways are not uniform in their approach to presenting services. Your gateway development team will be able to provide valuable input regarding the appropriate technologies to incorporate into your gateway. Recommended software from our experience working with a number of gateways is listed in the TeraGrid Wiki at Useful Software for XSEDE Science Gateways. This is an active area to which all may contribute their recommendations about frameworks, portlets, development environments and others.
Most gateways require user registration in the gateway; XSEDE resources are used via a single UNIX Community Account. With this strategy, each gateway user does not need to apply for an XSEDE allocation. This approach does mean that the gateway PI must track and report the usage of individual gateway users. Learn about the rationale and implementation for community accounts in the Developers section.
A smaller number of gateways allow their services to be used by XSEDE users who have individual allocations. There are techniques that can be used to facilitate this such as adding the gateway community account as a user on an individual allocation. Please contact email@example.com for additional information.
Where Will Your Web-based Gateway Be Hosted?
Most gateways are hosted at their home institution; however gateway hosting services are now available at some service provider (SP) sites. Consider XSEDE hosting if the infrastructure to maintain a secure Web environment does not exist on a local server or if when providing access to a large data collection hosted on XSEDE resources.
XSEDE Web Hosting
- Request the Indiana Quarry system with your allocation request
- XSEDE provides Apache and Tomcat server hosting and cgi-bin hosting.
Once a PI obtains an XSEDE allocation for XSEDE resources, he or she can create and connect a gateway to XSEDE. Ultimately, it is the PI's responsibility to make sure that the gateway development team follows XSEDE requirements and practices. There are a few specific obligations defined below.
Displaying the XSEDE Logo
Please display the XSEDE logo on your site. Many gateways place it in the footer. Please look on the Logo page for XSEDE Logos.
To Cite XSEDE in Publications
An acknowledgment of support from the XSEDE Science Gateways program should appear in a publication of any material, whether copyrighted or not, based on or developed with NSF-supported advanced computing resources.
The suggested language below can be tailored as appropriate for publications by the PI who sponsors the gateway: "This research was supported in part by the National Science Foundation through XSEDE resources provided by the XSEDE Science Gateways program" or "This work used the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by National Science Foundation grant number OCI-1053575 and the XSEDE Science Gateways Program." Users of your gateway will likely cite the gateway itself and XSEDE if they use XSEDE resources.
Citation of Science Gateways Initiative
To cite the TeraGrid Science Gateways initiative specifically, consider using one of these publications: Nancy Wilkins-Diehr, Dennis Gannon, Gerhard Klimeck, Scott Oster, Sudhakar Pamidighantam: TeraGrid Science Gateways and Their Impact on Science. IEEE Computer 41(11): 32-41 (2008) Nancy Wilkins-Diehr: Special Issue: Science Gateways - Common Community Interfaces to Grid Resources. Concurrency and Computation: Practice and Experience 19(6): 743-749 (2007) Nancy Wilkins-Diehr, Thomas Soddemann: Science gateway - Science gateway, portal and other community interfaces to high end resources. SC 2006: 16 (From The DBLP Computer Science Bibliography.)
- Quarterly User Counts: For a community account with activity in a given quarter, PIs will receive a request from the XSEDE Science Gateways Area Director for the number of gateway users who ran jobs on XSEDE during that quarter. This is an NSF requirement.
- Annual Summary of Activity: All recipients of XSEDE allocations are required to submit a summary of activity, whether the allocation is for an individual account or for a gateway.
Security & Accounting
PIs are responsible for securely operating their gateways and notifying XSEDE staff about any security incidents which have the potential to impact XSEDE. Proper accounting is required for fulfilling security and reporting requirements, as well as for justifying subsequent allocation renewals.
Because the gateway maintains control of community accounts, the gateway PI must ensure that NSF computational resources are used in a manner consistent with the award and that reasonable attempts have been made and tools installed to ensure appropriate usage. This includes monitoring of all gateway usage by the community. Specifics are detailed in the user responsibility form. Our experience in working with gateways has led to a list of recommendations for best practices to adopt in the implementation and management of your gateway to insure appropriate use. These recommended practices and a full listing of security and accounting requirements are outlined in the For Developers section.
For community accounts, risks may be greater and usage patterns less certain, so additional information is required to help prevent security incidents. This information must be submitted before jobs can be run externally via community accounts. Additionally, XSEDE Service Providers may choose to impose their own limitations on community accounts through operating system tools, restricted shells or other means.
Whether your gateway users are allowed to use XSEDE resources through a community account or through their own individual XSEDE allocation, the PI is responsible for collecting and reporting usage information for all individuals using XSEDE resources through the gateway and reporting quarterly or annually as part of a subsequent allocation request or in a final project report if a project has concluded.
When renewing Research allocations for your gateway, usage and papers published because of use of the resources are key factors in a successful proposal. Enumerating publications may be more challenging for a gateway to collect because of the disparate nature of the user community. Nevertheless, science successes can be important, both for XSEDE resource requests and for gateway funding requests as well.
Staying Informed: XSEDE User News
Subscription to the Gateway news category in XSEDE User News by at least one person on your gateway team is strongly encouraged. XSEDE staff post information about XSEDE activities and resources that impact gateways in this category. News postings can be received by e-mail, by RSS feed, or by monitoring the News Web site.
Your developers should subscribe to news categories for the XSEDE resources you intend to use.
Additional methods for staying informed and communicating with the gateway community are listed below. These opportunities are optional.
Communication with the Gateway Community
A number of opportunities are available to gateway PIs and their developers.
Teleconferences & Mailing List
All XSEDE personnel funded to work on gateway-related projects attend the bi-weekly XSEDE Gateways teleconference and occasional Access Grid meetings. These meetings ensure progress on funded activities and also provide a forum for information exchange, discussion of important gateway-related topics, and an opportunity for guest speakers to address topics of mutual interest. These telecons are open to all.
Gateway PIs and their staff are not required to participate in the calls; however, the calls can prove to be a great source of information. Similarly, Gateway PIs and their staff may subscribe to the mailing list. See the Community page for teleconference and mailing list information.
Workshops and conferences
PIs and developers may find attendance at Gateway-related workshops and meetings and annual XSEDE conferences when practical. Gateway- and XSEDE-related conferences are announced in XSEDE User News under the Science Gateway category. All PIs and their registered developers will automatically signed up for XSEDE User News.