Nuclear Science and Physics Programs Invest in a Major Expansion of the PDSF Cluster at NERSC

Eighty-two new servers and significant improvements in the overall computing and networking infrastructure will be added this fall to the PDSF (Parallel Distributed Systems Facility), a large Linux-based computer cluster that is currently operated as a partnership between three divisions of the Lawrence Berkeley National Laboratory: Nuclear Science, Physics, and the National Energy Research Scientific Computing Center (NERSC). Computing power will be expanded by 57 dual-processor nodes, and storage capacity will be tripled.

The PDSF serves the data-intensive computing needs of international high energy and nuclear physics experiments at the world's most powerful accelerator centers -- including the AGS/RHIC complex at Brookhaven National Laboratory (STAR, E895), CERN (NA49, ATLAS, ALICE), and Fermilab (CDF, E871) -- as well as neutrino detectors in Canada (SNO) and Antarctica (AMANDA), and other experiments that are expanding our knowledge of the universe.

"The large computing and storage capabilities of the PDSF play an essential role in helping researchers extract the physics from the terabytes of data that they produce," said Lee Schroeder, Director of Berkeley Lab's Nuclear Science Division. "For example, the STAR experiment is producing descriptions of nuclear collisions at the highest heavy ion collider energies ever achieved, and SNO's first physics results are helping to solve the 'solar neutrino problem' and provide further evidence for neutrino oscillations. These great physics results were made possible by the dedication of the NERSC/PDSF staff and their close collaboration with the research teams."

Projects that use the PDSF provide support for expansion and upgrades of the system. Currently the Berkeley Lab Nuclear Science and Physics divisions and the NERSC Center cover the costs of four NERSC Division employees who provide system administration and user support. The sponsoring divisions within the DOE Office of Science are Nuclear Physics, High Energy Physics, and Mathematical, Information, and Computational Sciences.

Clusters are assemblies of commodity computers designed and networked to operate as a single system. The PDSF currently consists of 281 processors with a theoretical peak processing capacity of 155 gigaflop/s and a total storage capacity of 7.5 terabytes. When the current expansion is completed in October 2001, the PDSF will have 395 processors, and the number of disk vaults will grow from 15 to 40, with a total 24 terabytes of shared storage.

Each new node will have two 1-gigahertz Pentium III CPUs, with 1 GB of memory per CPU. Gigabit Ethernet networking for the high-bandwidth compute nodes will enable the PDSF to run MPI jobs that require up to 50 nodes (100 processors). In addition, Gigabit Ethernet for the disk vaults will make it possible to take advantage of the server-side performance improvements of the 2.4 Linux kernel.

The STAR Collaboration is the largest user of the PDSF and is making the largest investment in its expansion: 16 disk vaults, 20 compute nodes, and 34 high-bandwidth compute nodes. The PDSF has become such an integral and indispensable part of the project's data analysis that they have changed the standard acknowledgment for all their publications to recognize NERSC's support.

Other research groups contributing to the current PDSF expansion are the CDF experiment at Fermilab, the Cosmic Microwave Background astrophysics program at Berkeley Lab, the ATLAS experiment at CERN, the DeepSearch supernova experiment, and the Relativistic Nuclear Collisions program at Berkeley Lab.

In addition to the new servers and disk vaults, the expansion will include added memory for some compute nodes as well as upgrades to console servers, switches, networking cables, disk drives, software, and miscellaneous tools -- all with the goal of continuing to provide a reliable, well-supported resource that meets its users' growing needs.