Artificial Intelligence Seminar
— 1:00pm
Location:
In Person and Virtual - ET
-
Newell-Simon 3305 and Zoom
Speaker:
OLATUNJI RUWASE
,
Lead and Co-founder, DeepSpeed Project, Microsoft Research
https://www.microsoft.com/en-us/research/people/olruwase/
DeepSpeed: Enabling efficient trillion parameter scale training for deep learning models
Deep Learning is driving unprecedented progress in a wide range of Artificial Intelligence domains, including natural language processing, vision, speech, and multimodal. Sustaining this rapid pace of AI revolution, however, requires practical solutions to the extreme demands of model scaling on the compute, memory, communication and storage components of modern computing hardware. To address this challenge, we created a deep learning optimization library called DeepSpeed to make distributed model training and inference efficient, effective, and easy on commodity hardware. This talk will focus on DeepSpeed optimizations for improving memory, compute, communication, and data efficiency of extreme-scale model training.
—
Olatunji (Tunji) Ruwase is the lead and co-founder of the DeepSpeed project at Microsoft. His broad industry and research background spans compilers, operating systems, and hardware accelerators. His current focus is on systems and convergence optimizations, and frameworks for efficient distributed training and inference of deep learning models. His research results on deep learning training, inference, and hyperparameter search are used in multiple Microsoft systems and products, such as Azure, Ads, Bing, Catapult, and HyperDrive. Tunji earned a PhD in Computer Science from Carnegie Mellon University under the guidance of Professor Todd Mowry.
In Person and Zoom Participation. See announcement.
Event Website:
http://www.cs.cmu.edu/~aiseminar/