Deep Learning at Scale Training, March 3-4, 2025

NERSC and NVIDIA are hosting a hybrid, hands-on Deep Learning at Scale training event on March 3-4 in Berkeley, CA . This training will help users explore distributed training for deep learning models on high-performance computing systems (specifically Perlmutter). The training will focus on building a large-scale deep learning model on a real scientific application (transformers for weather forecasting) and walk users through profiling tools and performance optimization on a single GPU, scaling to multiple GPUs (and nodes) through distributed training with data parallelism (along with tips and techniques to scale) as well as advanced parallelization for very large models with model parallelism. 

We will provide example code and datasets to allow attendees to experiment hands-on with optimized and scalable distributed training of our scientific deep learning model on Perlmutter. Due to the hands-on experiments on Perlmutter, the event attendance will be capped. However, all training material as well as the lecture recordings will be made available after the event. OLCF and ALCF users are welcome to attend. Training accounts will be provided if needed.

Logistics

This event will be hybrid. Onsite location (in B59, see visitor information for more details) and zoom link details will be shared soon. 

Agenda

All times are in Pacific time zone. Agenda below is tentative.

Registration

Registration will be on a first-come first-served basis and will be capped due to the hands-on nature of the event. Please fill out the registration form for attending.