NeuralSound: Learning-based Modal Sound Synthesis with Acoustic Transfer


Xutong Jin1

Sheng Li 1

Guoping Wang1

Dinesh Manocha2

1Peking University

2University of Maryland

Github Code
Supplementary Video


We present a novel learning-based modal sound synthesis approach that includes a mixed vibration solver for modal analysis and a radiation network for acoustic transfer. Our mixed vibration solver consists of a 3D sparse convolution network and a Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) module for iterative optimization. Moreover, we highlight the correlation between a standard numerical vibration solver and our network architecture. Our radiation network predicts the Far-Field Acoustic Transfer maps (FFAT Maps) from the surface vibration of the object. The overall running time of our learning-based approach for most new objects is less than one second on a RTX 3080 Ti GPU while maintaining a high sound quality close to the ground truth solved by standard numerical methods. We also evaluate the numerical and perceptual accuracy of our approach on different objects with various shapes and materials.


    title={NeuralSound: Learning-based Modal Sound Synthesis with Acoustic Transfer},
    author={Jin, Xutong and Li, Sheng and Wang, Guoping and Manocha, Dinesh},
    url = {},
    journal = {ACM Transactions on Graphics (SIGGRAPH 2022)},