Petascale Post-Doc Project a Supercomputing Success Story

The first post-doctoral research project centered at the Department of Energy’s (DOE) National Energy Research Scientific Computing Center (NERSC) is being credited with helping its participants advance their careers and boosting the state-of-the-art in high performance scientific computing software applications.

The Computational Science and Engineering Petascale Initiative – aka the Petascale Post-doc Project – was unveiled in 2009. The goal was to hire several post-doctoral researchers to work with science teams on research in key application areas that support DOE’s energy mission and encourage the use of NERSC supercomputers for energy research. The program was funded through the American Recovery and Reinvestment Act (ARRA).

In all, nine post-docs participated in the project, each for about two years: Brian Austin, Kirsten Fagnan, Christos Kavouklis, Jihan Kim, Bobby Liu, Filipe Maia, Praveen Narayanan, Robert Preissl and Rebecca Yuan. The program was centered at the Lab’s Oakland Scientific Facility, where NERSC resides, and overseen by principal investigator Alice Koniges of NERSC.

The post-docs collaborated with DOE-funded scientists on projects ranging from carbon capture and sequestration to fusion energy simulations, advanced accelerator modeling and geophysical imaging. They worked to increase the performance and science content of energy-related codes running on NERSC systems and provide feedback to NERSC users on application acceleration techniques, new programming models and algorithmic enhancement.

“Through interaction with the NERSC staff as well as the general population of NERSC users, the work produced a significant benefit to advanced computational research in the DOE community,” Koniges noted in a report summarizing the program’s achievements.  

The project made innovative contributions to key DOE research facilities, including Energy Frontier Research Centers (EFRC), projects funded by ARRA and mission-related science hubs.

For Kim and many of the post-docs, it was the first time they had worked at NERSC or any other supercomputing facility.

“I got involved with the petascale post-doc project after I saw a posting about it on the Internet and submitted an application,” said Kim, who worked with noted UC Berkeley professors Martin Head-Gordon and Berend Smit on two carbon capture projects. “I was interested in learning more about high-performance computing, and it seemed like a great opportunity for me.”

He credits the project with helping him land a faculty position in the Department of Chemical and Biomolecular Engineering at KAIST in South Korea, where he is currently working. “I could not have procured this position without my experience within the petascale post-doc project,” he said.

“When I became aware of the postdoctoral program at NERSC, I realized that it would be an excellent opportunity to run simulations on one of the most powerful supercomputers in the world,” said Kavouklis, who collaborated with Phillip Colella, director of Berkeley Lab’s Applied Numerical Algorithms Group, to develop a new version of the method of local corrections (MLC) code (developed by Colella) for the numerical solution of Poisson’s equation on 3D structured grids. He continues to work with Colella’s group in the Computational Research Division at Berkeley Lab.

Kavouklis and his colleagues credit the program with having a key impact on their careers.  While some have remained in academic research, several have gone on to work in the commercial side of HPC. Narayanan is with nVidia, Priessl was with IBM and is now involved with an HPC startup, Liu works for Google in Shanghai and Yuan is doing data analysis for Bank of America.

“This project has decisively made scientific computing the cornerstone of my career,” said Maia, who also worked on two projects during the petascale post-doc program – one assisting Stefano Marchesini of the Advanced Light Source in developing software for data analysis of ptychography experiments, the other converting a linear solver used in geological problems from CPU to GPUs. He is now on a tenure track at Upsala University in Sweden.

“This experience definitely made the difference between whether I remained in chemistry or transitioned into the more computational side of computational science,” said Austin, who has a PhD in theoretical chemistry but worked with Jonathan Wurtele and Ji Qiang at Berkeley Lab’s Center for Beam Physics to model a next-generation light source. “It helped me realize that I wanted to work more on the computational side of things.” In fact, Austin now works full-time at NERSC as a member of the Advanced Technology Group.

Similarly, Fagnan’s experience with the petascale project not only brought her to NERSC but paved the way to her current position as a bioinformatics consultant with Berkeley Lab’s Joint Genome Institute (JGI).

“Without the project, I probably wouldn’t have had this opportunity,” she said. “My skill set prior was more in the applied math area, but through the petascale project I got to run some cool calculations on tens of thousands of processors, which was great because I had never had the chance to do that before and it gave me credibility with JGI. One of the things we are trying to transition JGI from is single nodes or cores to more parallel workflow, and we are using my background in higher performance computing to help them make that transition.”

Click here to read the full report, which includes detailed descriptions of each post-doc project, milestones and key findings.