ICVGIP2012

ICVGIP2012Speech training aids

Visual feedback of the articulatory efforts during acquisition of speech production by a hearing-impaired child.

Display of articulatory effort using LPC-based analysis of speech signal

• Oral cavity: fixed length tubular sections.

• LPC analysis of windowed speech frames

>> LPC reflection coefficients >> Section area ratios >> Section areas, assuming constant glottis-end area >> Vocal tract shape [Wakita, 1973]

>> Display of the articulatory efforts not visible on speaker's face.

Introduction

ICVGIP2012

ICVGIP2012

Problem: Errors due to variation in glottis-end area during speech production [Wakita,1979] .

Proposed solution•Acquisition of speech as audio and facial image as video.•Using mouth opening area estimated from the video as the reference area of the lip-end section, for scaling of the area ratios obtained from LPC analysis of simultaneously acquired speech signal [Nayak et al., 2012] .

Investigation A technique for estimation of the mouth opening, without errors caused by teeth and tongue between the lips• Contrast enhancement with multi-threshold binarization • Connected component detection

ICVGIP2012

ICVGIP2012

Processing steps

iv) Horizontal opening

v) Vertical opening: segmentation, multi-threshold

binarization, connected component detection

vi) Det. of inner lip boundaries vii) Mouth opening area calculation

i) Input frame ii) Face sub-image iii) Mouth sub-image [Viola & Jones, 2004] [Hsu et al., 2002]

ICVGIP2012

ICVGIP2012

Test resultsTest results

•Test material: video recordings of vowels /a i u/ of 12 male speakers.

•Scatter plot of estimated values & values obtained manually

•Corr. coeffi.: 0.91