NERSC Delivers 59.9 Petabytes of Storage with Cutting-Edge Technology

NERSC's High Performance Storage System (HPSS) can now hold 59.9 petabytes of scientific data — equivalent to all the music, videos or photos that could be stored on approximately 523,414 iPod classics filled to capacity.

This 37-petabyte increase in HPSS storage was made possible by deploying cutting-edge technologies — the SunT10000 generation B-tape drive, which holds one terabyte of data on a single cartridge, and three Sun Slimline 8500 tape libraries, which add 15,000 new cartridge slots to the system.

Scientific Data on Demand: When a user requests HPSS data, a robot arm (bottom left) finds the cartridge using a bar code to verify the data inside. The robot then mounts the carridge in a tape drive, and streams the data back to the user. Typical tape mounts take between 30 and 90 seconds to complete.

“Since NERSC's Cray XT4 system, Franklin, went into production in March 2008, we've seen a 50 percent increase in new data archived to HPSS,” says Jason Hick, who heads NERSC's Mass Storage Group. “The new tape technology allows us to fill the ever-increasing demand for scientific archive space, with a negligible increase in our power consumption and very little impact on our floor space budget.”

According to Hick, one SunT10000 cartridge can hold the equivalent of all the data stored on three to five cartridges built with technology from 10 years ago. In addition to being cost and space efficient, tape storage is also a very green solution for saving the ever-increasing amount of scientific data generated as technology improves.

“At NERSC, scientists store all of the information that they are not currently analyzing on tape for later use. Since tape cartridges sit in a library slot offline, there is no power associated with that,” says Wayne Hurlbert, also of the Mass Storage Group. “Tape technology allows us to increase our storage capacity without increasing our power bill.”

According to Nick Balthaser, the Mass Storage Group worked hard over the past few months to put the new tape drive and library technology into production use without impact to the users. Now they face the “Herculean task” of migrating or rewriting scientific data currently archived on over 40,000 different cartridges onto the new tape technology. When complete, the data will utilize approximate 11,000 new cartridges. 

“The migration of existing data to the new media will take several years to complete, but will enable us to reclaim valuable floor space in the NERSC computer room,” says Balthaser. “It is exciting to work in a field of computer science that handles double the demand every two years without requiring increased cost, power, or space.”