National Energy Research Scientific Computing Center Accepts New IBM Supercomputer

BERKELEY, Calif. -- The U.S. Department of Energy's National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory today announced that the first phase of its new IBM RS/6000 SP system has met a demanding set of performance benchmarks and is now ready for full use by researchers across the nation.

As part of the acceptance testing, a select group of computational scientists at national laboratories and universities were given early access to the machine to thoroughly test the entire system. Those researchers noted the high performance and ability to scale problems on the SP, which is providing useful scientific results in such areas as climate modeling, materials science and physics research.

"The performance of this supercomputer -- and the fact that DOE-supported scientists are already putting it to productive use -- demonstrates the Department of Energy's continuing leadership in computational science," said Energy Secretary Bill Richardson. "Once the full machine is installed at NERSC later this year, thousands of researchers from universities and DOE laboratories will have access to the most powerful unclassified computing center on earth. They'll use this incredible resource to develop powerful new simulation tools for modeling and understanding human health, for developing new sources of energy, protecting our environment and understanding fundamental aspects of the physical world."

A key area of research by NERSC users is simulating combustion, with the goal of designing automobile engines which use less gasoline and emit fewer pollutants -- as well as eventually saving up to $31 billion annually in energy-associated costs in the United States.

The interior of an internal combustion engine is a hot, dirty and potentially dangerous environment for conducting experiments -- and building and modifying engine components for testing is both time-consuming and expensive. With accurate combustion modeling on supercomputers like the IBM SP, scientists can study the entire process and make incremental changes in the model system to test new ideas for studying internal combustion engines, as well as industrial boilers used to generate electricity. Besides saving energy, improvements in combustion could reduce annual emissions of carbon dioxide by 505 million tons.

The first phase of NERSC's IBM system consists of an RS/6000 SP with 304 POWER3 SMP nodes with two processors per node. In all, Phase I has 512 processors for computing, 256 gigabytes of memory and 10 terabytes of disk storage for scientific computing. The system has a peak performance of 410 gigaflop/s, or 410 billion calculations per second.

Phase II, slated for installation no later than December 2000, will consist of 152 16- processor SMP nodes, utilizing an enhanced POWER3 microprocessor. The entire system will have 2,048 processors dedicated to large-scale scientific computing and another 384 processors devoted to system tasks. The system will have a peak performance capability of more than 3 teraflop/s, or 3 trillion calculations per second. The second phase will be installed in Berkeley Lab's new scientific facility currently under construction in Oakland, Calif.

"We are committed to providing NERSC's national user community with the most advanced high-performance computing resources, and this partnership between Berkeley Lab and IBM ensures that the system met the very demanding performance criteria we defined at the outset," said Berkeley Lab Director Charles Shank. "Our priority is to ensure that the system will be capable of handling the day-in and day-out large-scale scientific computing needs of scientists across the country."

NERSC serves 2,500 researchers at national labs, universities and industry across the nation who are working on Department of Energy-funded programs such as combustion, climate modeling, fusion energy, materials science and computational biology. NERSC is also home to a Cray T3E-900 supercomputer, three Cray SV1 computers and a Cray J90se machine and provides its users with a High Performance Storage System capable of storing 750 terabytes of data.

Here are some early results from users who were allowed access to test NERSC's IBM RS/6000 SP as part of the acceptance testing:

• Researchers at the National Center for Atmospheric Research in Colorado ran the Parallel Climate Model (PCM), which simulates interaction of the atmosphere, ocean and sea ice, to evaluate changes in the earth's climate due to human activities, as compared/opposed to natural climate variability. To do this, the group plans to integrate climate data from several centuries, a process which will require thousands of processing hours on the new computer.
• A scientist from Ames Laboratory in Iowa used the computer to help answer a fundamental question in semiconductor surface research -- how different materials behave on the surface of silicon. Calculations performed on the supercomputer provided a detailed atomistic picture which is difficult to obtain from experiments and complements the experimental studies in obtaining a full understanding of this fundamental question. The scientist noted that the Phase I IBM SP ran the code 15 percent faster than NERSC's current Cray T3E supercomputer, and NERSC's Phase II IBM will be more than seven times faster than Phase I.
• Physicists at Argonne and Los Alamos national labs and the University of Illinois, Urbana-Champaign, used the IBM SP to calculate the properties of nuclei (up to 9 protons and neutrons) with realistic interactions using quantum Monte Carlo techniques, which find approximate solutions to problems by means of random sampling. The method uses the best available models of two- and three-nucleon interactions and gives results accurate to 1 percent for these forces. Some of their new scientific results have already been submitted for publication in a physics journal.
• Climate researchers at Oak Ridge National Laboratory have used the IBM system to determine the best method of running a massively parallel version of the Community Climate Model (CCM). The information they compile will be used in designing future versions of the CCM and other components of the DOE coupled climate models.

As part of the purchase contract, NERSC is working with IBM to develop and implement improved ways to make large-scale systems effective platforms for scientific discovery. One step is developing computer-utilization benchmarks and methods to assess and improve the effectiveness of the SP system in a production environment. While the theoretical peak performance of supercomputers can be amazingly fast, that capability does not always represent real-world computing. To ensure that the new NERSC system is well suited to the workaday world, NERSC and IBM developed an Effective System Performance (ESP) benchmark for the new computer. This set of tests will measure how well the SP delivers scientific work under a realistic workload.

"Although some computing centers describe their system's performance in terms of theoretical peak computing, we look at our systems in terms of how much they can enhance our clients' ability to solve large-scale scientific problems," said Bill Kramer, head of NERSC's High Performance Computing Department and leader of the IBM procurement effort. "That's the real measure of performance in our view."

During the acceptance period, NERSC and IBM worked to implement new functionality that improves the performance of the system for large-scale science in a way that will minimize the transition for scientists to the Phase II system. For example, NERSC is using IBM's new Global Parallel File System, GPFS, for all 10 terabytes of user data.

"NERSC is the first site to commit to use GPFS for all its data and to such a scale," Kramer said. "By wisely introducing new technology at this time, NERSC will provide higher performance overall, as well as making the move to the full system easier for the user community. Furthermore, by working with IBM to implement this functionality early and on a large system, NERSC continues to provide leadership in large-scale computing to the entire community."