acoustic reliability estimations for distant-speech recognition
TRANSCRIPT
![Page 1: Acoustic Reliability Estimations for Distant-Speech Recognition](https://reader034.vdocuments.net/reader034/viewer/2022052619/555e0216d8b42a3f618b5539/html5/thumbnails/1.jpg)
Acoustic Reliability Estimationsfor
Robust Distant‐Speech Recognition
Cristina Guerrero Flores
University of TrentoFBK ‐ SHINE Research Group
![Page 2: Acoustic Reliability Estimations for Distant-Speech Recognition](https://reader034.vdocuments.net/reader034/viewer/2022052619/555e0216d8b42a3f618b5539/html5/thumbnails/2.jpg)
Outline
1 ProblemContext and state of the art
2 Envisioning a SolutionKey intuition General IdeaExpected outcomes
3 Research PlanOngoing and subsequent work
![Page 3: Acoustic Reliability Estimations for Distant-Speech Recognition](https://reader034.vdocuments.net/reader034/viewer/2022052619/555e0216d8b42a3f618b5539/html5/thumbnails/3.jpg)
Machine Hearing
Problem [1/5] | Solution | Research Plan
Speech Recognition
![Page 4: Acoustic Reliability Estimations for Distant-Speech Recognition](https://reader034.vdocuments.net/reader034/viewer/2022052619/555e0216d8b42a3f618b5539/html5/thumbnails/4.jpg)
Problem [2/5] | Solution | Research Plan
Distant‐Speech Recognition
![Page 5: Acoustic Reliability Estimations for Distant-Speech Recognition](https://reader034.vdocuments.net/reader034/viewer/2022052619/555e0216d8b42a3f618b5539/html5/thumbnails/5.jpg)
State of the Art1) Automatic Speech Recognition (ASR)
Problem [3/5] | Solution | Research Plan
G. Potamianos et al. Automatic Speech Recognition in CHIL. Fifth European Conference on Speech, 1997.
2) Acoustic SceneAnalysis
Approaches
![Page 6: Acoustic Reliability Estimations for Distant-Speech Recognition](https://reader034.vdocuments.net/reader034/viewer/2022052619/555e0216d8b42a3f618b5539/html5/thumbnails/6.jpg)
Voice Enabled Smart‐Home
Problem [4/5] | Solution | Research Plan
![Page 7: Acoustic Reliability Estimations for Distant-Speech Recognition](https://reader034.vdocuments.net/reader034/viewer/2022052619/555e0216d8b42a3f618b5539/html5/thumbnails/7.jpg)
Problem
Acoustic scene descriptionhas been exploitedonly to a limited extent in distant‐speech recognition.
Problem [5/5] | Solution | Research Plan
![Page 8: Acoustic Reliability Estimations for Distant-Speech Recognition](https://reader034.vdocuments.net/reader034/viewer/2022052619/555e0216d8b42a3f618b5539/html5/thumbnails/8.jpg)
INTEGRATION
Rationale
Acoustic Scene AnalysisSource localizationSource separationSpeaker identificationAcoustic event detectionSpeech enhancement
Robust ASR Reliability Estimations
Problem | Solution [1/4] | Research Plan
1
2
3
1
![Page 9: Acoustic Reliability Estimations for Distant-Speech Recognition](https://reader034.vdocuments.net/reader034/viewer/2022052619/555e0216d8b42a3f618b5539/html5/thumbnails/9.jpg)
SourceSelection
Information FusionModule 1
Information FusionModule 2
CandidateFramework
Problem | Solution [2/4] | Research Plan
CMCM
CM
CM
CMi
![Page 10: Acoustic Reliability Estimations for Distant-Speech Recognition](https://reader034.vdocuments.net/reader034/viewer/2022052619/555e0216d8b42a3f618b5539/html5/thumbnails/10.jpg)
SourceSelection
Information FusionModule 1
Information FusionModule 2
Expected Outcomes
A different approach for the improvement of distant‐speech recognition
Multi‐level framework relying on acoustic scene information and reliability estimations
Robust CMs for different system components
Problem | Solution [3/4] | Research Plan
CMCM
CM
CM
CMi
![Page 11: Acoustic Reliability Estimations for Distant-Speech Recognition](https://reader034.vdocuments.net/reader034/viewer/2022052619/555e0216d8b42a3f618b5539/html5/thumbnails/11.jpg)
Solution
Use existing solutions as fundamental blocks of a coherent overall structurebased on statistical techniquesand confidence measures.
Problem | Solution [4/4] | Research Plan
![Page 12: Acoustic Reliability Estimations for Distant-Speech Recognition](https://reader034.vdocuments.net/reader034/viewer/2022052619/555e0216d8b42a3f618b5539/html5/thumbnails/12.jpg)
Roadmap
Problem | Solution | Research Plan [1/3]
II Year Integrate Location CM
RecognitionMulti‐Mic & Multi‐Room
Study CM Phase II(Acoustic Scene Analysis)
Study and Design ‐Fusion Techniques
I YearRecognition ExperimentsSingle‐Mic & CM ASR
Exp‐Study CM Phase I (ASR)
Study State of the Art
III Year Final design and implementation of Multi‐Source Information Fusion Modules
Evaluation of thewhole framework
01 02 03 04 05 06 07 08 09 10 11 12
ReportQualifying
Report forICASSP‐IEEE
months
Doctoral Courses
Initial Experiments on Fusion
Data Simulation
![Page 13: Acoustic Reliability Estimations for Distant-Speech Recognition](https://reader034.vdocuments.net/reader034/viewer/2022052619/555e0216d8b42a3f618b5539/html5/thumbnails/13.jpg)
Ongoing Work – CM as a Cue
Confidence Measures (CM) ‐ in Automatic Speech Recognition
Recognition metrics (e.g. WER)‐ in Sound Source Location Systems
Outcomes: CMs that express the reliability of the outputData collection (different acoustic conditions)Technique for the identification of # of sources
Problem | Solution | Research Plan [2/3]
![Page 14: Acoustic Reliability Estimations for Distant-Speech Recognition](https://reader034.vdocuments.net/reader034/viewer/2022052619/555e0216d8b42a3f618b5539/html5/thumbnails/14.jpg)
Research Plan
Gradually introduce challenges.Exploit pertinent technologiesand evaluate their impact in the proposed framework.
Problem | Solution | Research Plan [3/3]
![Page 15: Acoustic Reliability Estimations for Distant-Speech Recognition](https://reader034.vdocuments.net/reader034/viewer/2022052619/555e0216d8b42a3f618b5539/html5/thumbnails/15.jpg)
Summary
Objective: Novel distant‐speech recognition approach
Proposal: A framework that exploits the synergy of ASR & Acoustic Scene Analysis.Key element: Reliability estimations.
Strategy:Explore challenges individually.
![Page 16: Acoustic Reliability Estimations for Distant-Speech Recognition](https://reader034.vdocuments.net/reader034/viewer/2022052619/555e0216d8b42a3f618b5539/html5/thumbnails/16.jpg)
QUESTIONS
Cristina Guerrero [email protected]
Acoustic Reliability Estimationsfor
Robust Distant‐Speech Recognition