NERSC's Edison Supercomputer to Retire after Five Years of Service

The National Energy Research Scientific Computing Center’s Edison supercomputer, located at Lawrence Berkeley National Laboratory, opened for production use on January 14, 2014. Now, after more than five years of providing supercomputing capability to Department of Energy projects, the system will be retired on May 13, 2019.

A workhorse of a computer that is beloved by NERSC users, Edison has supported hundreds of research projects and enabled numerous critical scientific discoveries - everything from mapping the adolescent universe and assisting with weather/climate predictions to providing what was the fastest calculations in the early days of machine learning (see sidebar). 

Named for the famous inventor Thomas Alva Edison, the 30-cabinet Cray XC30 system features 5,586 compute nodes, 24 12-core Intel "Ivy Bridge" processors per node, 64 GB of memory per node, and a 7.56 petabyte scratch filesystem. With its 134,064 processing cores it can process information at 2.57 petaflops/sec.

“Edison has been a mainstay for science at scale for the DOE’s Office of Science computational workload,” said Richard Gerber, NERSC high performance computing department head and senior science advisor. “The system provided a first-of-its-kind internal high-speed network and, coupled with state-of-the-art processors and ease of use, enabled groundbreaking discoveries across all fields of interest to the Office of Science, from environmental and materials science to chemistry, cosmology, genomics, traditional and alternative energy, and more.” 

This supercomputer was one of the first Cray Cascade systems, with a design that features the Cray Aries Dragonfly high-speed network. This novel network topology enables faster communication between compute nodes, particularly those nodes that are physically far away from each other. Aries is also used in NERSC’s latest supercomputer, Cori.

“In our Hopper system (retired in 2015), all of the cabinets were connected in a 3D loop, so if somebody needed to communicate from one end of the machine to the other the data would need to hop from several cabinets to get there, which would slow down the computer significantly,” said Tina Declerk, the technical lead when Edison was first being built.

The Aries Dragonfly network helps overcome this problem, she added. Each cabinet has three chassis, each of which has a backplane, so every node is connected to every other node in the cabinet. The chassis are organized in pairs, and each chassis pair has a connector to every chassis in another cabinet, as well as with each other. “This creates a shorter communication delay between nodes at each end of the system,” Declerk explained.

A NERSC user favorite since its inception, Edison will be missed by researchers from around the globe. The center currently supports some 7,000 users who have depended on the system’s reliability and fast, single-thread compute nodes to speed their scientific productivity. Comments from NERSC’s most recent annual user survey included “Edison is awesome!” and “Edison is one of the best machines in the world.”

Users have also loved Edison for the continuity of its programming model from previous generations of HPC systems. “It is likely one of the last of its kind that we’ll see in large-scale HPC facilities as we make way for a new generation of manycore, accelerator, and specialized architectures that offer the potential for greater application performance growth within limited power budgets,” said Katie Antypas, NERSC division deputy and data department head.

Edison’s test system, login nodes, and part of the cooling system will be saved for future projects and systems, while the rest of the equipment will be returned to Cray as part of the NERSC-9 agreement. While NERSC would have loved to keep running Edison longer, the system needed to be removed to make way for the power upgrade for the NERSC-9 Perlmutter system. Due to be delivered in 2020, Perlmutter will be a heterogeneous Cray Shasta system comprising both CPU-only and GPU-accelerated cabinets. 

NERSC is a DOE Office of Science user facility.