XSEDE Science Successes
Comet Surpasses 10,000 Users
Comet Surpasses 10,000 Users
February 15 webinar to highlight SDSC resource capabilities
Published February 2, 2017
Comet, the petascale supercomputer at the San Diego Supercomputer Center (SDSC), an Organized Research Unit of UC San Diego, has easily surpassed its target of serving at least 10,000 researchers across a diverse range of science disciplines, from astrophysics to redrawing the "tree of life".
In fact, about 15,000 users have used Comet to run science gateways jobs alone since the system went into production less than two years ago. A science gateway is a community-developed set of tools, applications, and data services and collections that are integrated through a web-based portal or suite of applications. Another 2,600 users have accessed the high-performance computing (HPC) resource via traditional runs. The target was established by SDSC as part of its cooperative agreement with the National Science Foundation (NSF), which awarded funding for Comet in late 2013.
"Comet was designed to meet the needs of what is often referred to as the ‘long tail' of science – the idea that the large number of modest-sized computationally-based research projects represent, in aggregate, a tremendous amount of research that can yield scientific advances and discovery," said SDSC Director Michael Norman, principal investigator for the Comet project.
Comet, which went into operation in mid-2015,has been one of the most widely used supercomputers in the NSF's XSEDE (Extreme Science and Engineering Discovery Environment) program, which provides researchers with an advanced collection of integrated digital resources and services.
SDSC will hold a webinar on February 15 to provide a detailed overview of Comet's capabilities and system upgrades. First-time and current users are invited to attend. More information about the webinar can be found here.
"Now in its second year of serving the national research community, Comet is exceeding our expectations and we encourage new users to learn more about how Comet can support their research," said Norman. "Feedback from our current user base – both anecdotally and through their expressed use on the system, as well as examining the data we've been collecting – underscores a strong need for systems such as Comet that serve what we call the ‘99 percent' of the research community."
In addition to Comet's design, its allocation and operational policies are geared toward rapid access, quick turnaround, and an overall focus on scientific productivity. Comet also features large memory nodes, GPUs and local flash, which taken together, provide a highly usable and flexible computing environment for a wide range of domains.
Providing ‘Science Gateways' for Researchers
Surpassing the 10,000-user milestone in less than two years of operations is due in large part to researchers accessing Comet via science gateways, which provide scientists with access to many of the tools used in cutting-edge research – telescopes, seismic shake tables, supercomputers, sky surveys, undersea sensors, and more – and connect often diverse resources in easily accessible ways that save researchers and institutions time and money.
Science gateways make it possible to run the available applications on supercomputers such as Comet so results come quickly, even with large data sets. Moreover, browser access offered by gateways allows researchers to focus on their scientific problem without having to learn the details of how supercomputers work and how to access and organize the data needed.
In mid-2016, a collaborative team led by SDSC was awarded a five-year $15 million NSF grant to establish a Science Gateways Community Institute to accelerate the development and application of highly functional, sustainable science gateways that address the needs of researchers across the full spectrum of NSF directorates. The award was part of a larger NSF announcement in which the agency committed $35 million to create two Scientific Software Innovation Institutes (S2I2) that will serve as long-term hubs for scientific software development, maintenance and education.
"It's possible to support gateways across many disciplines because of the variety of hardware and support for complex, customized software environments on Comet," said Nancy Wilkins-Diehr, an associate director of SDSC and co-director of XSEDE's Extended Collaborative Support Services. "This is a great benefit to researchers who value the ease of use of high-end resources via such gateways."
One of the most popular science gateways across the entire XSEDE resource portfolio is the CIPRES science gateway, created as a portal under the NSF-funded Cyberinfrastructure for Phylogenetic Research (CIPRES) project in late 2009. The gateway is used by scientists to explore evolutionary relationships by comparing DNA sequence information between species.
In 2013, SDSC received a $1.5 million NSF award to extend the project to make supercomputer access simpler and more flexible for phylogenetics researchers. Typically, about 200 CIPRES jobs are running simultaneously on Comet.
"The scheduling policy on Comet allows us to make big gains in efficiency because we can use anywhere between one and 24 cores on each node," said Mark Miller, a bioinformatics researcher with SDSC and principal investigator of the CIPRES gateway. "When you are running 200 small jobs 24/7, those savings really add up in a hurry."
To date, the CIPRES science gateway has supported more than 20,000 users conducting phylogenetic studies involving species in every branch of the "tree of life". The gateway is used by researchers on six continents, and their results have appeared in more than 3,000 scientific publications since 2010, including Cell, Nature, and PNAS.
I-TASSER: A New Protein Structure Gateway Available via Comet
In late 2016, a new science gateway called I-TASSER (Iterative Threading ASSEmbly Refinement), developed by researchers at the Zhang Lab at the University of Michigan's Medical School, began accepting users. I-TASSER is a hierarchical approach to protein structure and function prediction. Structural templates are first identified from the Protein Data Bank using LOMETS (Local Meta-Threading-Server), an on-line web service for protein structure prediction. Full-length atomic models are then constructed by iterative template fragment assembly simulations. Finally, function insights of the target are derived by threading the 3D models through the protein function database called BioLiP.
Since October 2016, I-TASSER has been accessed via Comet – the only resource within the XSEDE portfolio to do so – by more than 8,000 unique users, according to Yang Zhang, a U-M professor of computational medicine and bioinformatics as well as biological chemistry, and the I-TASSER's principal investigator. In total, I-TASSER currently has more than 76,000 registered users from 130 countries.
"With the increasing requests from the community for protein structure and function modeling, one of the major bottlenecks of the I-TASSER Server has been the limit in supporting computer resources of our laboratory that was originally funded by the Department of Computational Medicine and Bioinformatics at the University of Michigan," said Zhang. "The generous grant of computing resources from XSEDE is very helpful in improving the capacity of the I-TASSER system to serve the broader biomedical community by providing faster and higher quality simulations of protein models."
Including I-TASSER, 29 science gateways are available via XSEDE's resources, each one designed to address the computational needs of a particular community such as computational chemistry, phylogenetics, and the neurosciences. SDSC alone has delivered 77 percent of all gateway cycles since the start of the XSEDE project in 2011.
As an Organized Research Unit of UC San Diego, SDSC is considered a leader in data-intensive computing and cyberinfrastructure, providing resources, services, and expertise to the national research community, including industry and academia. Cyberinfrastructure refers to an accessible, integrated network of computer-based resources and expertise, focused on accelerating scientific inquiry and discovery. SDSC supports hundreds of multidisciplinary programs spanning a wide variety of domains, from earth sciences and biology to astrophysics, bioinformatics, and health IT. SDSC's Comet joins the Center's data-intensive Gordon cluster, and are both part of the National Science Foundation's XSEDE (Extreme Science and Engineering Discovery Environment) program.
Image: Ben Tolo, SDSC
- XSEDE Resources, Trinity Enable Non-Human Primate Reference Transcriptome Resource to Support Study of Genes in Our Closest Relatives
- Turtle Tree of Life
- Region 1 Champions meet at Idaho National Laboratory
- Crash test simulations expose real risks
- NSF supports development of new arctic maps
- How was the planet Earth formed?
- Exploring Large Data for Scientific Discovery
- XSEDE Value Added
- Scholars program helps realize dream
- Making sense of cyberinfrastructure
- XSEDE15 Wrap Up
- Bioinformatics Scripts Solutions
- XSEDE15 Plenary Panel
- Polymer Potential
- The Future of NSF Advanced Computing Infrastructure
- 2015 International Summer School on HPC Challenges
- A Catalyst for Complexity
- As Austin Grows So Does Its Traffic Woes
- The University of Tennessee, Knoxville, Wins Second Place in an International Student Supercomputing Competition
- PSC Receives NSF Award for Bridges Supercomputer
- Innovative New Supercomputers Increase Nation's Computational Capacity and Capability
- Exploring Competitive Balance
- A Direct Bridge
- The Dopamine Transporter
- XSEDE Supercomputers Laid the Foundation for an Unprecedented Simulation of Cosmological Evolution
- Big Data Needs Big Funding
- XSEDE helps create a more effective way to assemble genomic information
- Of Micelles and Machines
- XSEDE Allocation System to Receive Makeover
- Internet2: Advancing Science in the Age of Big Data
- XSEDE User Portal At Your Fingertips: Mobile App
- Researchers Study Air Pollution
- Dan Stanzione: New Executive Director at TACC
- People of XSEDE: Campus Champions - Preaching the HPC Gospel
- XSEDE and Blue Waters Go Supernova
- Two at a Time
- Show Him the Money
- Cosmic Slurp
- Turning Salt into the Unknown
- Looking Inside Images
- Farming the Wind
- Breaking out of the Digital Graveyard
- The Mechanism of Short-term Memory
- Open Science and Industry Collaboration
- XSEDE, Prace Call for Requests of Joint Support
- XSEDE Wins HPCWire Award
- Shields to Maximum, Mr. Scott
- The Ultimate Timekeeper
- Blue Waters, XSEDE sign collaborative agreement
- People of XSEDE - Outreach programs set XSEDE apart
- Wrangler Reels in Award
- The Great Comet: NSF awards $12 Million Grant to SDSC to deploy Comet
- Meet the Gribbles
- 2013 Nobel Prize in Chemistry winners bring HPC to the lab
- XSEDE helps create a more effective way to assemble genomic information
- XSEDE facilitates large-scale image analysis to understand diseases
- XSEDE announces new campus briding services and tools
- XSEDE, NSF Release Cloud Survey Report
- XSEDE13: Programming Competition Allows Students to "Geek Out" and Gain Crucial Skillsets
- Katlin Thaney gave XSEDE13 Keynote: Gateways for Open Science
- XSEDE13 conference selects best papers, posters visualizations and more
- XSEDE13 speaker tells how turbulence simulations help make movie magic
- XSEDE13 Plenary Talk: Accelerating Brain Research with Supercomputers
- Invited speakers announced for Extreme Scaling Workshop - Heterogenous Computing
- XSEDE13 speaker LeManuel "Lee" Bitsóí: Democratizing Scientific Research
Read more about Bitsóí's talk at this year's conference
- More than 70 students from 4 continents gain HPC skills at fourth annual Summer School
- Registration opens for Extreme Scaling Workshop 2013
- Campus Champions Fellows Named
- Campus Champions program reaches 200 members
- Rock Snot Genomics: University of Texas researchers use advanced sequencing and TACC's Ranger supercomputer to uncover origin of common algae
- Experiencing some turbulence: Researchers Take on One of Physics' Most Important and Enduring Problems
- Register now for Virtual School summer courses on data-intensive and many-core computing
- XSEDE seeks a Scientific Workflow Specialist for Extended Collaborative Support Service
Applications are due May 31, 2013
- XSEDE13 schedule now available online
- Students from high school to grad school levels invited to participate in programming contest at XSEDE13 high performance computing conference
- SDSC's Gordon enables discoveries in the study of genetics Read about Gordon's role in pinpointing the genetic patterns underlying autism-spectrum disorders, schizophrenia and similar brain conditions.
- XSEDE, National Computational Science Institute offer summer workshops for educators
- XSEDE13 Student Day applications due May 15 High school and undergraduate students get hands-on experience in computational science and interact with expert researchers
- XSEDE upgrades to Internet2's 100G Network
- XSEDE13 Registration now open!
- Get to know XSEDE Staff XSEDE Allocations Manager Ken Hackworth: The Man, The Myth, The Legend
- Two sponsors commit to XSEDE13 conference: Cray and Intel .
- Texas Unleashes Stampede
- Swirling Secrets-Understanding the turbulence of gases
- Blacklight helps researchers develop better materials for carbon capture
- Journey to the limits of spacetime
- Students invited to participate in XSEDE13 Multiple ways for high school, undergraduate, and graduate students to get involved; funding support available.
- XSEDE Call for Humanities, Arts and Social Science ProjectsIf you and your collaborators need to access to large collections of digital data, need more computer power, or require substantial storage capacity and computing power – please share it with XSEDE.
- XSEDE needs your feedback! If you received an invitation to complete the 2013 User Satisfaction Survey, please take 10 minutes today to share your comments about the XSEDE user experience.
- XSEDE deploys Globus Online for data transfer The first official software service on XSEDE has been accepted for production deployment
- The Stampede Era Begins XSEDE supercomputer now operational and available to the national open science community
- Call for ParticipationInternational Summer School on HPC Challenges in Computational Sciences
- XSEDE, European Grid Infrastructure seek collaborative use cases
Deadline extended to March 8!
- XSEDE offers free online parallel computing course Learn to use parallel computers more efficiently and productively
- NICS makes the top of Green500 list XSEDE partner recognized for energy-conscious high-performance computer, Beacon
- XSEDE's John Towns appointed to Compute Canada board of directors Board includes leaders in industry, academia, and computational research
- STILL ACCEPTING RESPONSES to Cloud Use Survey from XSEDE, NSF All researchers encouraged to respond and help shape future of cloud computing in XSEDE
- Make room for Stampede: TACC expands data center for new supercomputer
Read more about the new data center at TACC
See TACC Deputy Director, Dan Stanzione describe the new center
- SDSC welcomes Gordon supercomputer as a research powerhouse
Read more about SDSC's Gordon
- Campus Bridging Early Adopter Program issues Call For Proposals to be submitted Dec. 1-9
Read more about the program
- XSEDE12 announced -- first conference of Extreme Science and Engineering Discovery Environment
Read more about XSEDE12
- PSC, SGI Team Up on Shared-Memory Supercomputer
Read more about PSC's shared-memory supercomputer
- Pittsburgh Supercomputing Center Wins High-Performance Computing Award
Read more about PSC
- Blacklight Goes to Work at the Pittsburgh Supercomputing Center
Read more about Blacklight
- Ranger supercomputer's lifespan extended one year as part of NSF XD initiative.
Read more about Ranger
- Kraken set to deliver 2 billionth CPU hour, sustains 96 percent utilization
Read more about Kraken
- TACC Offers New, Broader Computational Biology Software Stack to Open Science Community.
Read more about biology software stack
- ACM launches new Special Interest Group on High Performance Computing. Join by Nov. 18 for special rate.
Read more about the new SIGHPC
- 'What Are You Working on Today,' Ranger, Jaguar and iForge?
Read more about TACC's Ranger supercomputer
Read more about ORNL's Jaguar supercomputer
Read more about NCSA's iForge supercomputer
- Adventures with HPC Accelerators, GPUs and Intel MIC Coprocessors
Read more about experiences with new hardware
- Developing Scientific Computing Communities
Read more about development efforts
- Indiana University to create the National Center for Genome Analysis Support, which will be integrated with XSEDE resources
Read more about the NCGAS at IU
- Scientists use XSEDE/TeraGrid resources to determine how shock waves move through solids
Read more about 'super-elastic shock waves'
- XSEDE upgrades network
Read more about the XSEDE upgrade
- Richard Tapia, Rice University mathematician and professor and member of XSEDE outreach team, receives National Medal of Science
Watch the Oct. 21 webcast
Read more about Tapia's award
Learn more about Richard Tapia
- Stampede's comprehensive capabilities to bolster U.S. open science computational resources
Read more about Stampede
Watch a video of Jay Boisseau, director of TACC, discussing Stampede
- SDSC announces scalable, high-performance data storage cloud
Read more about SDSC cloud
- Appro and SDSC Gordon supercomputer to provide up to 35M IOPS
Read more about SDSC's Gordon
- Dr. Barry Schneider from the National Science Foundation to describe XSEDE in the Oklahoma Supercomputing Symposium keynote, Oct. 11-12
Read more about Dr. Schneider's keynote
Go to symposium site
- Students research solar cells with HPC
Read more about HPC and solar research
- Seeing Is Believing: Extreme Digital visualization and data analysis resources help researchers derive insights from massive data sets
Read more about Extreme Digital
- New "Memory Advantage Program" on Blacklight at the Pittsburgh Supercomputing Center
Read more about PSC's MAP
- XSEDE project brings advanced cyberinfrastructure, digital services, and expertise to nation's scientists and engineers
Read more about XSEDE
- Watch the John Towns video
- How XSEDE will facilitate collaborative science
Read more about XSEDE and collaboration