NERSC, Cray, Intel to Collaborate on Next-Generation Supercomputer

The U.S. Department of Energy’s (DOE) National Energy Research Scientific Computing (NERSC) Center and Cray Inc. announced today that they have signed a contract for a next generation of supercomputer to enable scientific discovery at the DOE’s Office of Science (DOE SC).

Lawrence Berkeley National Laboratory (Berkeley Lab), which manages NERSC, collaborated with Los Alamos National Laboratory and Sandia National Laboratories to develop the technical requirements for the system.

The new, next-generation Cray XC supercomputer will use Intel’s next-generation Intel® Xeon Phi™ processor –- code-named “Knights Landing” -- a self-hosted, manycore processor with on-package high bandwidth memory and delivers more than 3 teraFLOPS of double-precision peak performance per single socket node. Scheduled for delivery in mid-2016, the new system will deliver 10x the sustained computing capability of NERSC’s Hopper system, a Cray XE6 supercomputer. 

NERSC serves as the DOE SC’s primary high performance computing (HPC) facility, supporting more than 5,000 scientists annually on over 700 projects. The $70 million plus contract represents the DOE SC’s ongoing commitment to enabling extreme-scale science to address challenges such as developing new energy sources, improving energy efficiency, understanding climate change, developing new materials and analyzing massive data sets from experimental facilities around the world.

“This agreement is a significant step in advancing supercomputing design toward the kinds of computing systems we expect to see in the next decade as we advance to exascale,” said Steve Binkley, Associate Director of the Office of Advanced Scientific Computing Research. “U.S. leadership in HPC, both in the technology and in the scientific research that can be accomplished with such powerful systems, is essential to maintaining economic and intellectual leadership. This project was strengthened by a great partnership with DOE’s National Nuclear Security Administration.”

To highlight its commitment to advancing research, NERSC names its supercomputers after noted scientists. The new system will be named “Cori” in honor of bio-chemist and Nobel Laureate Gerty Cori, the first American woman to receive a Nobel Prize in science.

Technical Highlights

Cori the supercomputer will have over 9300 Knights Landing compute nodes and provide over 400 gigabytes per second of I/O bandwidth and 28 petabytes of disk space. The contract also includes an option for a “Burst Buffer,” a layer of NVRAM that would move data more quickly between processor and disk, allowing users to make the most efficient use of the system while saving energy. The Cray XC system features the Aries high-performance interconnect linking the processors, which also increases efficiency. Cori will be installed directly into the new Computational Research and Theory facility currently being constructed on the main Berkeley Lab campus.

“NERSC is one of the premier high performance computing centers in the world, and we are proud that the close partnership we have built with NERSC over the years will continue with the delivery of Cori – the next-generation of our flagship Cray XC supercomputer,” said Peter Ungaro, president and CEO of Cray. “Accelerating scientific discovery lies at the foundation of the NERSC mission, and it’s also a key element of our own supercomputing roadmap and vision. Our focus is creating new, advanced supercomputing technologies that ultimately put more powerful tools in the hands of scientists and researchers. It is a focus we share with NERSC and its user community, and we are pleased our partnership is moving forward down this shared path.” 

The Knights Landing processor used in Cori will have over 60 cores, each with multiple hardware threads with improved single thread performance over the current generation Xeon Phi co-processor. The Knights Landing processor is “self-hosted,” meaning that it is not an accelerator or dependent on a host processor. With this model, users will be able to retain the MPI/OpenMP programming model they have been using on NERSC’s previous generation Hopper and Edison systems. The Knights Landing processor also features on-package high bandwidth memory that can be used either as a cache or explicitly managed by the user.

“NERSC’s selection of Intel's next-generation Intel® Xeon Phi™ product family – codenamed Knights Landing – as the compute engine for their next generation Cray system marks a significant milestone for the broad Office of Science community as well as for Intel Corporation,” said Raj Hazra, Vice President and General Manager of High Performance Computing at Intel. “Knights Landing is the first true manycore CPU that breaks through the memory wall while leveraging existing codes through existing programming models. This combination of performance and programmability in the Intel Xeon Phi product family enables breakthrough performance on a wide set of applications. The Knights Landing processor, memory and programming model advantages make it the first significant step to resolving the challenges of exascale.”

Application Readiness

To help users transition to the Knights Landing manycore processor, NERSC has created a robust Application Readiness program that will provide user training, access to early development systems and application kernel deep dives with Cray and Intel specialists. 

“We are excited to partner with Cray and Intel to ensure that Cori meets the computational needs of DOE’s science community,” said NERSC Director Sudip Dosanjh. “Cori will provide a significant increase in capability for our users and will provide a platform for transitioning our very broad user community to energy-efficient, manycore architectures. It will also let users analyze large quantities of data being transferred to NERSC from DOE’s experimental facilities.”

As part of the Application Readiness effort, NERSC plans to create teams composed of NERSC principal investigators along with NERSC staff and newly hired postdoctoral researchers. Together they will ensure that applications and software running on Cori are ready to produce important research results for the Office of Science. NERSC also plans to work closely with Cray, Intel, DOE laboratories and other members of the HPC community who are facing the same transition to manycore architectures.

“We are committed to helping our users, who represent the broad scientific workload of the DOE SC community, make the transition to manycore architectures so they can maintain their research momentum,” said Katie Antypas, NERSC’s Services Department Head. “We recognize some applications may need significant optimization to achieve high performance on the Knights Landing processor. Our goal is to enable performance that is portable across systems and will be sustained in future supercomputing architectures.”

DOE’s Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov.