|Thesis abstract: |
The research topic addressed by my doctoral work concerns the extraction of high-level descriptors from multimodal signals. In particular, the assessment of the instrumental quality of musical instruments from multimodal signals (vibrational, acoustic, timbral etc.) is a relevant example of this problem. Violins (and bowed instruments), in fact, represent a particularly complex case of study. They have been the subjects of intensive research for centuries and, yet, some of their acoustic and vibrational properties are not fully understood. The purpose of my work is to develop a methodology for extracting high-level descriptors concerning instrumental quality, timbral quality, etc. using machine intelligence, applied to multimodal signals coming from: audio recordings (timbral acquisition), geometric information (e.g. 3D scans), acoustic signals (e.g. reproduced soundfield, radiance pattern, etc.), vibrational measures, material analysis signals (e.g. spectroscopy data). The scientific laboratories of the Politecnico di Milano and the University of Pavia at the Violin Museum offer a chance to gather precious data to approach these problems.
My main focus, so far, has been the extraction of high-level timbral descriptors based on machine learning. In addition to that, I have focused on using tools of the Semantic Web (e.g. ontologies) to build a formal representation of the terms used by humans to describe the sound.Recently, a project was started joining the efforts of Politecnico di Milano and Università degli Studi di Pavia, in collaboration with the Museo del Violino. The purpose of this project is to develop a scientific knowledge on the violin making world, with a multimodal approach: several analyses are conducted, like audio recordings, 3D scans, vibrational measures, spectroscopies and so on.
In this context, my main focus is the analysis of timbre perception. By means of machine learning techniques I want to develop a system to analyse the timbre of an instrument and determine how the human listener will perceive it. I am making use of tools of the Semantic Web like ontologies to build a formal representation of the terms used by humams to describe the sound.