Physically-based sound rendering enriches 3D animation. However, it is difficult to make an object with a given shape produce a specific sound using physically-based sound rendering because the user would need to define appropriate internal material distribution. To address this, we propose an example-based method to design physically-based sound for a 3D model. Our system optimizes the material distribution inside the 3D model so that physically-based sound rendering produces sounds similar to the target sounds specified by the user. A problem is that modal analysis required for this optimization is prohibitively expensive. In order to run the optimization at an interactive rate, we present fast approximate modal analysis that enables three orders of magnitude acceleration of the eigenproblem computation compared to standard modal analysis for an elastic object. It consists of data-driven online coarsening of the mesh and hierarchical component mode synthesis with efficient error correction. We demonstrate the feasibility of the method with a set of comparisons and examples.
An example-based interactive design framework for rendering the physically-based sound of a 3D model using material optimization.
The user first provides a 3D surface model to the system as input. Next, the user selects a few sample positions
on the model surface, and assigns corresponding sound clips
that define the target sounds to be rendered when the positions are struck.
The system then optimizes the material distribution inside the model so that physically-based sound simulations yield the expected sounds.
Our optimization algorithm consists of the precomputation and runtime. An iteration of our optimization procedure at runtime consists of 3 steps. First, the system computes modal analysis to obtain the vibrational property of the object. Second, it computes the similarity score between the simulated sounds of the object and user specified target sounds. According to this similarity score, the system updates the material distribution inside the object to minimise the cost.
To execute the optimization at an interactive rate, we also present a novel fast approximate modal analysis method that achieves three orders of magnitude acceleration com- pared to the standard modal analysis. Our technique consists of data-driven finite element coarsening of the mesh (DDFEM*) and hierarchical component mode synthesis with efficient error correction (HCMS+EC).
In the below example, we assigned the metal sound to position 1 (plate) and wooden sound to position 2 (handle), and optimized it. As the result, the two spectrograms after the optimization closely resemble each target sound. In addition, even if a different position from the one assigned is struck, the sound characteristics of the target sounds near the position is produced in a physically plausible manner.
Finally, we demonstrated a complicated scenario involving several objects. In the scene, the sounds of the all objects are designed by our system, and all sounds are triggered automatically by rigid body simulations. It took 30 minutes for us to design sound of all the objects. This shows our approach provides the user to intuitive design workflow of the sound of an object with a set of examples.
・Kazuhiko Yamamoto, and Takeo Igarashi. 2016. Interactive Physically-Based Sound Design of 3D Model using Material Optimization.
ACM SIGGRAPH / Eurographics Symposium on Computer Animation (SCA), 2016. [PDF]
・山本 和彦, 五十嵐 健夫. 2016. 物理ベースサウンドデザインのための例示ベースインタフェース.
WISS, インタラクティブシステムとソフトウェアに関するワークショップ, 2016 [PDF]
@inproceedings{10.5555/2982818.2982849,
author = {Yamamoto, Kazuhiko and Igarashi, Takeo},
title = {Interactive Physically-Based Sound Design of 3D Model Using Material Optimization},
year = {2016},
publisher = {Eurographics Association},
booktitle = {Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation},
pages = {231–240},
numpages = {10},
series = {SCA '16}
}
Kazuhiko Yamamoto – www.yamo-n.org
mail: yamo_o(at)acm.org
twitter: @yamo_o