Generating Precise 3D Drummer Motion from Drum Sounds Alone
This research presents a diffusion-based system that synthesizes realistic, precise drumming motion from audio. It separates skeletal integrity from drumstick-target precision and is designed to gener
Drumming is a particularly challenging task in music-based character animation. The performance must make rapid acceleration look natural while ensuring that the drumsticks reach the correct contact points at exactly the right moments. Existing methods often rely on motion matching or MIDI input, limiting their ability to handle a wide range of real-world audio.
This paper proposes a generative diffusion framework for synthesizing a drummer's movements from audio. Its key feature is a dual-objective loss function designed to maintain human-skeleton consistency and natural full-body motion while controlling drumstick positions with centimeter-level precision.
The researchers used their own dataset and data augmentation strategies to help the model generalize to uncurated audio recorded in real-world environments. They also proposed new metrics for measuring the distance to contact points and evaluating the temporal alignment between audio and motion. Quantitative analysis and a user study were used to validate the quality of the generated movements.
The authors are Álvaro G. Iñesta, Mattia Ryffel, Amit H. Bermano, Robert W. Sumner, and Martin Guay, and DisneyResearch|Studios released the research on August 18, 2026. The paper was also included in the program for the 2026 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA).