Computer Music Seminar
— 3:00pm
Location:
4405
-
Reddy Conference Room, Gates Hillman 4405 and Zoom
Speaker:
GUS XIA
,
Assistant Professor of Machine Learning, and, DIrector, Music X Lab, Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)
http://www.musicxlab.com/members/gus/
Recent Progress in Multimodal Music Language Models: Hierarchical Modeling, Control, and AI Secretary
In this presentation, Gus will delve into three pioneering studies focused on developing multimodal language models for music AI. First, he will discuss whole-song generation through cascaded diffusion modeling, which integrates hierarchical music language structures into diffusion models for enhanced sample-efficient learning.
Second, he will introduce "coco-mulla," the inaugural "controlnet" fashion model that adeptly applies content-based controls to large-scale audio generation tasks. Lastly, Gus will present Flute X GPT, which introduces LAUI (LLM-Agent User Interface) as a potential next-generation Human-Computer Interaction (HCI) paradigm after Graphical User Interface (GUI). This system capitalizes on a nuanced understanding of both users and tutorial software to innovate customized music-learning experiences.
—
Gus Xia is an assistant professor of machine learning at MBZUAI, as well as an affiliated faculty at NYU Shanghai, Tandon, CILVR at the Center for Data Science, and MARL at Steinhardt. He received his Ph.D. in the machine learning department at Carnegie Mellon University (CMU) in 2016, and he was a Neukom Fellow at Dartmouth from 2016 to 2017. Xia’s research is very interdisciplinary and lies in the intersection of machine learning, HCI, robotics, and computer music.
Some representative works include interactive composition via style transfer, human-computer interactive performances, autonomous dancing robots, and haptic guidance for flute tutoring. Xia is also a professional Di and Xiao (Chinese flute and vertical flute) player. He plays as a soloist in the NYU Shanghai Jazz Ensemble, Pitt Carpathian Ensemble, and Chinese Music Institute of Peking University. In 2022, Xia and his students held a Music AI concert in Dubai. I
n Person and Zoom Participation. See announcement.
For More Information:
In Person and Virtual - ET