Zhijian Liu
Bio: Zhijian Liu is a Ph.D. candidate at MIT, advised by Song Han. His research focuses on efficient deep learning. He has developed efficient algorithms and systems for deep learning and applied them to computer vision, robotics, natural language processing, and scientific discovery. His research has been adopted by Microsoft, NVIDIA, Intel, and Waymo. He was selected as the recipient of Qualcomm Innovation Fellowship and NVIDIA Graduate Fellowship. He was also recognized as a Rising Star in ML and Systems by MLCommons and a Rising Star in Data Science by UChicago and UCSD. Previously, he received his B.Eng. degree from Shanghai Jiao Tong University.
Talk Title: Efficient Deep Learning with Sparsity — Algorithms, Systems and Applications
Abstract: Deep learning has catalyzed advancements across numerous scientific and engineering fields. It is also a driving force behind many successful real-world applications, such as autonomous driving, large language models, and generative AI. Despite its remarkable accomplishments, the computational requirements of deep learning have surged dramatically. For instance, the model size of large language models is increased from 0.11B parameters in GPT-1 to 175B in GPT-3 within three years, an over 1500X growth. However, the pace of hardware acceleration has slowed down in the post-Moore era. This raises a large (and potentially growing) gap between the supply and demand of AI computing.
In this talk, I will introduce my research efforts on efficient deep learning with sparsity. I will first talk about my efforts on developing efficient sparse algorithms and systems. I will mainly focus on TorchSparse that effectively translates the theoretical savings from activation sparsity to practical speedups. It is three times faster than the leading industry solution from NVIDIA. I will then delve into my efforts on incorporating sparsity into different applications, such as computer vision, robotics and high-energy physics. To conclude, I will provide a glimpse into some of my ongoing research projects and outline my long-term research vision.