NERSC and CRD Help Decipher Science from Compact Accelerator Simulations

3D Particle Surfing : Electrons are moving from left to right in this LOASIS experiment. The blue shells show the plasma wake, while bunches of surfing electrons are shown as green and yellow. Green reveals low-energy particles, and yellow shows high-energy. The simulation was conducted in the VORPAL framework, and visualized with VisIt and VizSchema.

Scientists use many different tools to understand the universe. While telescopes offer valuable insights about interactions between stars, planets and galaxies; particle accelerators give researchers a better understanding of the basic particles that make up these structures, as well as the fields and forces that hold them together. The bigger the accelerator, the more energy subatomic particles can pick up, which leads to higher-intensity impacts that expose ever more basic particles, fields and forces. However, bigger is also costlier, and scientists worry that accelerators built with today's technology may simply be unaffordable.

That's why members of the Laser Optics and Acceleration System Integrated Studies (LOASIS) program, led by Wim Leemans, in the Lawrence Berkeley National Laboratory's (Berkeley Lab) Accelerator and Fusion Research Division, and collaborators at the Tech-X Corporation are looking into laser-plasma accelerators (LPAs) — machines that can be compact enough to sit on a tabletop and produce accelerating electric fields up to 10,000 times greater than conventional accelerators.

As part of their research, the LOASIS team is collaborating with analytics and visualization experts from the Computational Research Division (CRD) and the National Energy Research Scientific Computing (NERSC) Division to refine their understanding of the fundamental physics driving the next-generation accelerator. The Berkeley Lab’s Computing Sciences organization includes both CRD and NERSC.

“Our ultimate goal is to understand how plasma wakes form and how they accelerate particles. Experts from the NERSC and CRD visualization and analytics groups are helping us achieve this by providing the algorithms and tools that give us insights into the physics in very large simulations where conventional tools break down,” says Cameron Geddes, of the LOASIS team. “By understanding how electrons are injected and accelerated, we can gain a better understanding of what needs to be done to refine the beam.”

Science of Speeding Electrons

A conventional accelerator like the Stanford Linear Collider (SLC) in Palo Alto, Calif., which stretches about two-miles long, relies on radiofrequency fields in large metallic cavities to accelerate electrons. LPAs, on the other hand, speed up electrons by getting them to “surf” on a wave of plasma, a gaseous stew of separated electrons and ions.

Scientists create a plasma wave, or wake, by shooting a laser pulse into the plasma. The laser pushes the relatively light electrons out of its path, while the heavier ions stay in place. Eventually, the negatively charged electrons are pulled back toward the positively charged ions, creating a plasma wave that supports a longitudinal electric field. If the laser is like a motorboat pushing through the water, the plasma wave is like the wake behind the boat.

Similar to the way human surfers paddle to match the speed near the peak of an ocean wave, electrons are injected into the high-density area of the plasma wave to pickup speed. As ocean surfers drop in front of the wave to ride it, earth's gravitational field pulls them down, enabling them to be accelerated. In LPAs, it is the wake's longitudinal electric field that pulls the electrons and allows them to surf near the speed of light.

According to Geddes, the high electric fields produced by the plasma waves are what makes LPA technology so promising. Because of those high fields, the size, and hence the cost, of accelerators could be significantly reduced using LPAs.

Perfecting the Beam

Particle Tracks: Colors in this image show longitudinal momentum. Blue represents relatively low-enery particles, while red reveals highest-energy. The particles are injected into the wake at low energy ( far left), gain energy as they surf the plasma wave (middle), and the energy again decreases as a result of dephasing, or the process where electrons slow down as they outrun the wave. As they accelerate, the particles undergo transverse oscillation in the Y-PY plane.

“Acceleration rate is only one measure of a good accelerator; the number of particles in the beam and their spread in energy and in angle, or focus, are also critical,” says Geddes.

A few years ago LOASIS and other groups' experiments demonstrated for the first time that a narrow energy spread in the LPA's particle beams resulted from a complex interplay of electron trapping, loading of the wave, and dephasing when the electrons outrun the wave. Now, scientists at LOASIS and Tech-X are working to refine the beam by coupling computer simulations at NERSC with experiments to gain a better understanding of the physical properties behind the injection and acceleration of electrons.

Geddes notes that LPA simulations are particularly challenging because of the widely separated scales that must be resolved. The wavelengths of the lasers are sub-micron in size, while the accelerating electrons surf over centimeters of plasma, soon to be meters in new experiments. In addition, the simulations also have to keep track of millions of particles to simulate the plasma. Because of these complexities, large simulations can require millions of hours on tens of thousands of processors. Most of the larger LOASIS simulations were run on NERSC's CrayXT4 system, called Franklin, which currently contains 38,128 processors.

Each run produces terabytes of data, and to develop tools and techniques to analyze this mountain of information, LOASIS researchers and their collaborators at Tech-X enlisted the help of CRD and NERSC analytics and visualization experts.

“The most important part of our job is to work closely with researchers and get a sense of how they work. Our goal is to develop tools and techniques that they can intuitively use,” says Oliver Ruebel of the CRD Visualization Group.

Visualizing the Physics

LOASIS Simulation Snapshot: this image reveals the formation of compact particle beams. The high-density particle beams, detected using clustering techniques, are identified in white. Blue, green, yellow and red show the energy of the particles; blue is relatively low energy, while red represents high-energy electrons.

To help LPA researchers extract pertinent physics information from multi-terabyte datasets and create three-dimensional images of the laser and plasma wake for analysis, the visualization team relied on VisIt, an interactive parallel visualization application.

Because the accelerator simulations were conducted in the VORPAL framework, the first order of business was to load the VORPAL files into VisIt using the VizSchema markup and plugin. Researchers at Tech-X developed the VORPAL and VizSchema.

The analytics and visualization experts then incorporated the FastBit indexing software into VisIt, which enabled the computers to automatically scan millions of plasma particles in the simulation and track each particle's trajectory. After FastBit, they applied clustering and machine-learning techniques to pinpoint areas where electrons were injected onto the wave and where they were bunching into narrow beams. This information gives researchers like Geddes valuable insights into the physics at play, which in turn helps them to refine the electron beam.

FastBit was developed by researchers in CRD's Scientific Data Management Group, led by Kesheng “John” Wu, and won an R&D 100 Award in 2008.

“By applying these techniques, scientific data exploration and analysis tasks that previously took hours to complete now only take a few seconds,” says Daniela Ushizima of the CRD Visualization Group and NERSC Analytics Team.

As the researchers run more simulations and produce even more data, Ruebel and Ushizima believe that the new challenge will be to make this process even more automatic. They are both looking into new techniques that can differentiate between the particle injection site and the electron beams that form as the particles surf the wake. As the scientists learn more about the physics that drive LPAs, the analytics experts can develop tools that will more accurately identify and characterize the electron beams.

“In an ocean of millions of simulation particles, the electron beam is only a few thousand particles bunching together. They are extremely difficult to spot,” says Ruebel. “As more simulations are conducted, researchers will have to sift through even more data. Our hope is to make life easier for them by letting computers comb through the data automatically, so that the scientists can focus on the analysis.”

In addition to those mentioned above, other visualization and analytics contributors include Gunther Weber and Prabhat of CRD and NERSC, as well as Marc Durant, Paul Hamill, and Svetlana Shasharina of Tech-X. Simulation contributors include Berkeley Lab's Eric Esarey and Estelle Cormier-Michel; as well as David Bruhwiler, John Cary, Ben Cowan, and Peter Messmer of Tech-X. The VisSchema and VORPAL teams at Tech-X developed the data plugins and simulation software used in the LPA experiments.

The VORPAL framework was developed by researchers from Tech-X and partially funded by Small Business Innovation Research and by the Community Petasacle Project for Accelerator Science and Simulation (ComPASS) project, lead by Panagiotis Spentzouris. ComPASS is part of DOE's Scientific Discovery through Advanced Computing (SciDAC) program.

The Berkeley Lab Computational Research Division's Visualization Group is partially supported by DOE's Office of Advanced Scientific Computing Research, by the NERSC program through the NERSC Analytics Team, and by the SciDAC Visualization and Analytics Center for Enabling Technologies (VACET), which is led by Wes Bethel.