MUSCLE eTeam: Semantic from Audio
(MUSCLE NoE | All e-teams)
- Evaluation of different features sets for audio, music and instruments classification, in several applications
- Events “quasi”-object detection
- Dynamic optimization of DSP parameters in music rendering, using features extraction.
We will investigate and develop special methods for extracting
features sets in audio and musical signals. We will also try to make a
sort of "quasi"-object detection in audio signals. These extracted
features will be used for several applications as music/instruments
classification and retrieval (TU Wien, AUTH, ISTI-CNR), dynamic
optimization of a digital signal processing algorithm for an
enhancement of recorded music (ISTI-CNR, CUED Cambridge), etc. We will
also work on the extension of a C++ library for audio processing and
synthesis, which could be used for optimizing the
feature extraction processes and, more in general, for audio/musical
applications (ISTI-CNR, UNIS).
The results will be:
The result of this e-team in the first stage would thus provide a kind of
practical state-of-the-art showcase of the capabilities and competences
on feature extraction and semantic annotation within MUSCLE.
- prototype systems for music and instrument tones classification,
retrieval, rendering to be used in commercial, educational,
- Jointly written articles
- Student exchanges
Contribution of partners
- ISTI-CNR (Graziano Bertini, Massimo Magrini, Leonello Tarabella)
Signal processing activity started during the '70s for biomedical
applications (neurology, cardiac pacemaker-patent '66) and, later,
industrial and computer music systems (audio terminal AT2, '75),
firstly using analog/digital mixed technology and then continued with
DSP technology (half '80s).Research in audio fields now concerns:
methods and systems for audio/music real-time processing and synthesis
and touch-less interfaces for computer music performances. The topics
related to the proposed e-team mainly regard:
1. Evaluation and improvement of a digital signal processing algorithm
for enhancement of recorded music, in order to adding more vivacity and
better transients perception.
2. Bandwidth extension for ipo-acusic people for better sound quality
in music listening by special equalized gain curve and usual headphones
(tools for hearing statistical evaluation, design of suitable
analog/digital portable systems etc.).
3. Evaluation and improvement of pCM++ (pureCMusic), a SW framework for
audio/music synthesis able to manage input data from external
4. Improvement of gesture tracking algorithms based on video
acquisition, integrated in pCM++, for controlling real-time generated
- TU Wien - IFS (A. Rauber, T. Lidy)
We have active a number of activities in the field of audio analysis,
particularly feature extraction for audio retrieval, mostly for genre
classification (PlaySOM) - we are participating in the ISMIR MIREX
contest in these disciplines. Another focus topic, that would allow
some overlap and integration with the User Interface WP, is interfaces
to audio collections, based on such extracted features and subsequent
- AIIA - AUTH (C. Kotropoulos, G. Benetos)
The activity regards the automatic musical instrument classification of
isolated tones and sound segments by extracting timbral and MPEG-7
Audio features using Non-Negative Matrix factorization (NMF).
- CUED – Cambridge (S. Godsill, A. Cemgil)
Current area of researches relates probabilistic models and associated Bayesan numerical
methods for audio and music processing applications as musical score
transcription, source separation and audio restoration.
- UNIS – Surrey (B. Christmas)
Our current activities include the detection and recognition of
distinctive sounds in sports videos. This work also contributes
towards WP6 - automated combined analysis of audio and video in
sporting events. We have made available a public domain (LGPL) C++
library that includes a framework for integrating different types of
audio (& video) sources & outputs.
The focus of this e-Team is particularly on encouraging collaboration
between participants via exchange of tools as well as know-how
their specific audio analysis tools. The goal here is to evaluate the
suitability of the respective tools for the variety of heterogeneous
tasks addressed in the various labs. This will be achieved by a set of
short-term exchange visits in order to jointly work on processing the
data and study the results of the impact of the different approaches.
Tentative plan of activities
- Feature extraction for automatic transients enhancement:
looking for a set fo features, such us the energy variations
in sub-bands of the song (average of dynamic range variations, average
time of attack/release peak envelopes, rythms , genres etc..) and then
using this feature to set ARIA algorithm parameters .
a) Off line modality test (searching the features useful to set ARIA parameters)
b) For a case study we will try to add analysis functionality to
the “PlaySom” system, developing a special ARIA Plug-in for
classification/identification algorithms developed at AIIA-TH for feature extraction could be used on
sound retrival and also for some other applications (such as educational,
instrument training etc.)
- Wide band compensation of presbycusic people for a better quality hearing of music:
Definition of set of suitable audio/musical segments having
special features, to be used as test signals for the evaluation of the
music perception of hearing impaired people.
Development of a platform with SW/HW tools with special user interface
suitable for audiology specialists in order to carry out not-standard
(wide-band) tests on hearing impaired people, using the set of signals
- Porting and extension of pCM++ (a SW framework for audio/music synthesis and processing) for Mac OS X on Windows and Linux platforms.
The framework will be available as open source on the e-team site.
- Powerpoint Presentation given at the 4th Scientific Meeting in Istanbul.
- Partnership of ISTI audio team in a new EC Project MODEM
(Music Open Distance Exchange Model - Contract n.
I/05/B/F/PP-154059) regarding the development of a web based environment for exchanging
multi-track audio/musical signals. Beside ISTI other
Italian and EU University and private partners are involved. A "local
workshop" will be organized at ISTI in May. Some other Muscle E-team
partners could be officially involved in the system testing.
Istituto di Scienza e Tecnologie dell'Informazione, Consiglio Nazionale delle Ricerche
ISTI-CNR, Area di Ricerca CNR, Via G. Moruzzi, 1
56124 Pisa, Italy
Tel: +39 050 3153125 - +39 050 3153144 (lab.)
Mobile: +39 348 3972163
Fax: +39 050 3152810
Dept. of Software Technology and Interactive Systems
Vienna Univ. of Technology
Favoritenstr. 9 - 11 / 188
A - 1040 Wien