adapted representations of audio signals for music instrument recognition pierre leveau laboratoire...

Adapted representations of audio signals for music

instrument recognition

Adapted representations of audio signals for music


Pierre Leveau

Laboratoire d’Acoustique Musicale, Paris - France

GET - ENST (Télécom Paris), France

Pierre Leveau - ENST - LAM 2

SummarySummary

• Master Thesis: Music instrument recognition on solo performances with signal segmentation (transient part / release part)

• Ph. D. Thesis: Structured and sparse decompositions: application to audio indexing


Music Instrument Recognition Music Instrument Recognition

• Basic Scheme

Feature extraction

Training DB (manually indexed)

Classificationmodel

Comparison to the model

File to analyzeFeature

extractiondecision

Music instrument recognition on solo performances with signal segmentation (transient part / release part)


Feature ExtractionFeature Extraction

• Feature Extraction on frames of fixed size (30 ms)

Analysis Frames



Music Note SchemeMusic Note Scheme

time

energyEx: strong attack instrument



Interest of transients for Music Instrument Recognition

Interest of transients for Music Instrument Recognition

piano trumpet

cello flute



Chosen MethodChosen Method

• Signal segmentation into transient part / release part

• Approximation: fixed length transients

• Need of an automatic onset detection algorithm.

• Study of solo performances



Onset DetectionOnset Detection

• Detection function (ex: high frequency content, spectral difference, phase deviation…)

• Peak-picking



Evaluation of Onset DetectionEvaluation of Onset Detection

• Necessity of an reference onset database

• ROC Curves

good

det

ectio

ns %

false alarms %



Sound Onset LabelizationSound Onset Labelizationspectrogram

Signal plot

Sound listening and labels positioning

Reference Onset and Sound Databases



Onset DatabaseOnset Database

Annotation precision depending on the file type

Detection function evaluation must take it into account



Annotation precision: examples

Annotation precision: examples

trumpet

cello



Developed Detection Function

Developed Detection Function

Complex Spectral Difference:

Delta Complex Spectral Difference:

guitar violin



Detection Function comparison

Detection Function comparison

Tolerance window TROC = 100 ms TROC = Topt



Signal segmentationSignal segmentation

R RT T T R RT

Analysis Frames



Music Instrument recognition on transients - Results

Music Instrument recognition on transients - Results

Music instrument recognition only on transients implies:- big decrease of the learning database size- for a fixed duration of the test signal, less data to take a decision.

Results worse than for a recognition on all frames



PerspectivesPerspectives

• Increase the onset database size for a more robust evaluation

• Improve the robustness of the Onset detection algorithm

• Merge decisions on transients and steady part, compare to the classical static recognition.

• Select features adapted for each part of the notes.



Ph. D. ThesisPh. D. Thesis

Subject:

Sparse and structured decompositions: application to audio indexing

Under supervision of Gaël Richard (GET - ENST, Paris)

and Laurent Daudet (Laboratoire d’Acoustique Musicale, Paris)


SparseRepresentations

SparseRepresentations

• Classical representations: Orthogonal transform (ex: Fourier Transform, STFT, MDCT, Wavelet Transform…)

• Redundant representations:

x u

Sparse representations (only on N terms):

x ii0

N 1

ui RN

: Redundant dictionnary



Dictionary ExampleDictionary Example

DCW

C: MDCT basis (useful to represent tonal parts of signals)

W: DWT basis (useful to represent transient parts of signals)



AlgorithmsAlgorithms

• Matching Pursuit (and its variants): Greedy algorithms Based on an iterative search Faster algorithm needs a suboptimal search

• Molecular Matching Pursuit: Gives structured, perceptually relevant organizations

of the atoms (by grouping significant coefficients) Faster than standard MP Fast varying frequencies (ex: vibrato) cannot be

efficiently represented



Application to music instrument recognitionApplication to music


Signal Feature Extraction

Classical Music Instrument Recognition

Comparison to statistical models

Decision

Signal MMP Feature Extraction (which features?)

Comparison to statistical models

(which models?)

Decision

Music Instrument Recognition with sparse decomposition

features

featuresStructuredRepresentation



To be continued…To be continued…

Thank you for your attention.

adapted representations of audio signals for music instrument recognition pierre leveau laboratoire...

Documents

signal segmentation

francepierre leveau

release partapproximation

release partph

enst tlcom paris

paris franceget

ms troc

lamannotation precision