Advanced Insights Ep. 4: Google on 5th Epoch of Distributed Computing

Amin Vahdat, Engineering Fellow and VP of Machine Learning, Systems and Cloud AI at Google, and I delve into the transformative era of distributed computing. We explore the key characteristics, challenges and opportunities, as well as the future landscape. 0:00 – 0:30 Introduction 0:30 – 1:47 Amin’s Bio 1:48 – 2:53 Shared experience 2:53 – 3:34 Leapfrogging Innovation 3:35 – 4:59 1970 Blank Canvas 4:59 – 5:52 Innovation Impact and challenges 5:52 – 6:58 Supercomputing and Connecting People 6:58 – 7:33 Moore’s Law and the Personal Computing Era 7:33 – 7:47 Connected Computers 7:47 – 8:31 More Powerful PCs 8:31 – 9:07 2000s and Doubling Slows 9:07 – 11:14 Scale Out Takes Hold and Cluster-based Computing 11:14 – 12:28 Totality of Human Knowledge 12:28 – 13:37 Demand for More Computing 13:37 – 14:22 New Types of Accelerators 14:22 – 15:17 30 Seconds/Day = 10 Googles 15:17 – 15:50 5th Generation TPUs 15:50 – 16:17 Transformers Organizing Information 16:17 – 16:58 Model Growth 10X Annually 16:58 – 19:45 Specialized Accelerators for the Win? 19:45 – 20:55 Flavors of General Purpose 20:55 – 22:36 Power Challenge and Google’s 2030 power goals 22:36 – 25:13 Power Provisioning 25:13 – 26:52 Open Source vs Proprietary 26:52 – 27:58 Societal Infrastructure 27:58 – 29:21 Importance of Team Culture 29:22 – 30:30 Growth Mindset vs Static Thinking 30:31 – 31:55 Dangers of Village Mentality 31:56 – 32:34 Communicating Culture 32:35 – 33:45 Thank You Amin 33:46 – 34:29 Episode Wrap Up