Programming Environments and Models Group Addresses Shifting Needs of Users and HPC

The landscape of high performance computing (HPC) is always expanding. Technology shifts; developers and users approach application development and execution in new ways with cascading effects. As a leader in the field, the National Energy Research Scientific Computing Center (NERSC) must evolve as well. As of fall 2022, a new group at NERSC, the Programming Environments and Models (PEM) Group, has assembled to address the growing needs of the NERSC user community in the face of an increasingly complex HPC landscape.

Overseen by Acting Group Lead Brandon Cook, the mission of the PEM Group is to evaluate, implement, and support modern HPC programming models, development environments, and software runtimes that let scientists take full advantage of the opportunities enabled by NERSC’s state-of-the-art supercomputers.

In addition to supporting users on Perlmutter, NERSC’s newest system, the group is also working to help users maintain a common code base that works across diverse architectures like those at other DOE facilities and novel ones that will arise in the next decade. The team also works with language standards bodies and vendors to ensure they produce programming environments that support the HPC needs of scientists.

Working with Complexity

Perlmutter derives most of its computational processing capability from 7,168 high-end NVIDIA A100 GPUs (Graphical Processing Units). While GPUs offer robust computational performance, they require special programming approaches to realize that potential. With a broad variety of methods, approaches, languages, and frameworks is already in play, each with differing levels of support, that’s where the PEM group steps in.

“You need ways to program state-of-the-art hardware,” said Cook. “That’s where the programming models come into play: essentially all the different programming languages that scientists would use to put their algorithms into language that the computer can understand.”

In addition to the complexity of the systems, users approach NERSC with increasingly diverse and sophisticated ways of incorporating HPC into their workflows. The PEM group works to make sure the features they need are accessible and working seamlessly.

“In addition to traditional approaches to HPC, we are also leading the way with new modes of using HPC, with interactive notebooks via Jupyter, containers for portability and flexibility and APIs to programmatically combine computation with data movement both within and beyond NERSC,” said Cook. “Taken together with the scale that NERSC provides, that amounts to a much more complex environment.”

Integration

Yet as the field as a whole becomes more complex, there’s also an increasing need for consistency and communication across facilities and platforms. Research infrastructure is moving toward increased integration, signaling a need for points of commonality. Part of the PEM Group’s mission is to aid in making those connections seamless.

"Both supercomputers and the workflows that scientists want to use are becoming increasingly complex," said senior computing engineer Rollin Thomas, a member of the group. “Users now want to integrate places like NERSC into multi-facility workflows, like with the superfacility efforts. But this level of complexity and integration into a large shared environment raises a number of issues and challenges that many scientists are just not used to. That’s a collision course, but meeting it head-on is what PEM is all about."

Taken as a whole, the task of keeping up with the changing HPC landscape and ever-evolving user needs is a big challenge, added NERSC chief architect Nick Wright, but it’s a job that’s key to NERSC’s role as a driver of science.

“I’m very excited that NERSC has created the PEM group,” said Wright. “With the increasing complexity of users’ workflows and the broad array of software available today, it’s essential that NERSC deploys an integrated software stack to ensure our users remain productive.”

Preparing for the Future

NERSC installs a new supercomputer approximately every five years, a process that involves envisioning the needs and opportunities coming in the future and building an environment that can accommodate that vision. As such, even as the PEM group works to optimize existing NERSC systems for users, they’re also a key voice in planning for upcoming ones.

“We think a lot about what types of compilers and programming languages are supported,” said Cook. “And also, a lot of the underlying capabilities of the system that maybe aren’t immediately obvious or observable to users but are enabling and allow us to expand the suite of our offerings in terms of what kinds of services and APIs NERSC can provide.”

A Full Set of Programming Options

In the meantime, the PEM Group is working on a series of features that will improve the current and near-future user experience in ways both visible and not so visible.

“I think NERSC might be the first center to offer what I would call a full set of portable programming options,” said Cook. “You can bring all kinds of different codes and get them to work here on our machine. So you could bring your code that’s targeting the Aurora system at Argonne or Frontier at Oak Ridge, and we can support that here as well. I want to see a lot of that support hardened and improved so that all the options are baked in and easy to access on equal terms as best we can.”

Overall, the PEM Group is already ensuring that NERSC is as robust, usable, and ready for the future as it can be.

“The HPC landscape is changing rapidly, with new programming techniques, languages, and models available to scientific researchers,” said NERSC HPC department head Richard Gerber. “At the same time, vendor and community support for these tools, as well as legacy ones, is uncertain and evolving. The Programming Environments and Models Group formed to help NERSC make sure it is delivering the most productive development and runtime environments for our users and to ensure that vendors and standards bodies consider the needs of Office of Science researchers.”