Prashant Shenoy, (University of Massachusetts Amherst)- Data Centers, AI Workloads, and Efficiency: An Energy Perspective

Abstract: The exponential growth of cloud computing has been a defining trend of our time, fueled by rapidly growing demands from online and data-intensive workloads. Despite the end of Denard scaling, the cloud’s energy demand grew more slowly than expected over the past decade due to the aggressive implementation of energy-efficiency optimizations. However, the rise of AI workloads, which are often more resource-intensive than traditional cloud workloads, has led to rapid growth in data centers with power-hungry accelerators such as GPUs and TPUs, leading to a resurgence in the cloud’s energy consumption and a strain on our electric grids.

In this talk, I will provide an energy perspective on the challenges and opportunities in enhancing the efficiency and sustainability of cloud platforms in the face of rising AI demand. I will discuss how resource management techniques such as workload shifting can enhance the efficiency of cloud platforms by exploiting the spatio-temporal variability in grid demand, energy availability, and electricity prices. I will present initial directions in making current computing systems grid-friendly and discuss approaches for navigating performance, efficiency, and cost tradeoffs that arise in their operations. I will end with several open research challenges that the research community needs to tackle to make AI-driven cloud platforms grid-friendly and ensure their continued growth.