Paul Liang joins the MIT Media Lab's Program in Media Arts and Sciences (MAS) and MIT Schwarzman College of Computing with a shared appointment in MIT Electrical Engineering & Computer Science (EECS) as Assistant Professor, Artificial Intelligence (AI) and Human Experience starting September 1, 2024.

Paul Liang recently received his PhD in Machine Learning at Carnegie Mellon University, advised by Louis-Philippe Morency and Ruslan Salakhutdinov, and is visiting the Simons Institute at UC Berkeley this summer. During his PhD, he also spent time at the research labs of DeepMind, Facebook, Nvidia, and Google. Previously, he received an MS in Machine Learning and a BS with University Honors in Computer Science and Neural Computation from CMU.

Paul's past research has built the theoretical foundations, large-scale resources, and neural network architectures to learn from multisensory data. These approaches are widely used to train and evaluate today's multimodal generative AI models, and he has applied them to healthcare and robotics in collaboration with practitioners. He is a recipient of the Siebel Scholars Award, Waibel Presidential Fellowship, Facebook PhD Fellowship, Center for ML and Health Fellowship, Rising Stars in Data Science, and four paper awards at international conferences and workshops. Outside of research, he received the Alan J. Perlis Graduate Student Teaching Award for designing a new pedagogy for multimodal AI.

In this new position, Paul Liang will lead pioneering research at the intersection of artificial intelligence (AI) and human experience. 

He will have a shared appointment in both the Program in Media Arts and Sciences, administered by the MIT Media Lab (under the MIT School of Architecture and Planning) and the MIT Schwarzman College of Computing, contributing to the Department of Electrical Engineering and Computer Science which is jointly administered by SCC and the MIT School of Engineering. Liang will teach undergraduate and graduate courses related to AI and human experience, focusing on how AI can be designed to enhance human experiences and improve AI performance through adaptation to individuals. His research will delve into the social, cultural, emotional, and ethical dimensions of human-AI interactions.

"We are thrilled to welcome Paul Liang to the MIT Media Lab," states Dava Newman, Director of the MIT Media Lab and Apollo Program Professor of Astronautics chair at MIT. "His pioneering research on multisensory AI, which integrates diverse sensory channels to enhance human experiences, exemplifies our commitment to innovative, interdisciplinary research. Paul's development of a taxonomy of key principles addresses the critically important area of understanding and mitigating social biases in language models. We eagerly anticipate the new frontiers he'll help us explore in shaping the future of AI and enriching human experiences."

Liang will supervise and mentor students, fostering their development in these cutting-edge areas. His role will also involve shaping the academic work and cultures of both MAS and SCC, ensuring they remain inclusive, equitable, and diverse. Liang’s teaching load will be divided between MAS and SCC-relevant curricula, covering a broad spectrum of AI-related topics. He'll produce original scholarly research, contributing to peer-reviewed publications that advance our understanding of AI and human experience. By collaborating with various departments and leveraging his expertise, Liang will help MIT maintain its leadership in AI research and its application to real-world human challenges. His dedication to interdisciplinary innovation and excellence will be crucial in driving forward the academic and research missions of both MAS and SCC.

"His emphasis on AI approaches to fuse multimodal sensory input resonates very well with the Media Lab, where we have a strong interest and deployment history in multimodal interfaces and systems," states Joseph A. Paradiso, Alexander W. Dreyfoos (1954) Professor and Associate Academic Head, Program in Media Arts and Sciences at the MIT Media Lab. "Paul’s presence with us will move these traditions forward into a new technical era.”

Paul's research group at the MIT Media Lab and EECS will focus on advancing multisensory AI to enhance human experiences. This group will have two thrusts:

1. Foundations of multisensory AI that can learn and interact with the world through integrating diverse sensory channels such as text, speech, audio, video, physical sensors, and physiological messages.
2. Impact on the human experience: AI for human physical, emotional, and social wellbeing, generative AI to augment human creativity, climate and environment sensing, and AI for new senses including music, art, smell, taste, IoT, and multimedia AR/VR.

"Most of today’s AI systems only perceive a narrow slice of the world," states Liang.

Collaborating with the diverse communities of the MIT Media Lab and EECS will unlock numerous possibilities for integrating music, art, cultures, smell, taste, and real-world sensors into the AI toolkit, enhancing its ability to deeply understand and elevate the human experience. 

"I am thrilled about the possibilities that lie ahead, come join us at the best place for multisensory AI research!"