cairo university faculty of computers and information
DESCRIPTION
Cairo University Faculty of Computers and Information. HMM Based Speech Synthesis Presented by Ossama Abdel-Hamid Mohamed. Agenda. Speech Synthesis HMM Based Speech Synthesis Proposed System Challenges. Speech Synthesis. What is speech synthesis? - PowerPoint PPT PresentationTRANSCRIPT
December 2006
Cairo UniversityFaculty of Computers and Information
HMM Based Speech Synthesis
Presented byOssama Abdel-Hamid Mohamed
2 HMM Based Speech Synthesis
Agenda
Speech SynthesisHMM Based Speech SynthesisProposed SystemChallenges
3 HMM Based Speech Synthesis
Speech Synthesis
What is speech synthesis?– Generating human like speech using computers.
Applications– Text To Speech.
– Conversation systems.
– Speech to speech translation.
– Concept to speech.
Systems built since late 1970s.– MITTALK 1979
– Klattalk 1980
4 HMM Based Speech Synthesis
Speech Synthesis, Cont.
Challenges:– Intelligibility.
– Naturalness.
– Pleasantness.
– Emotions.
5 HMM Based Speech Synthesis
Speech Synthesis, Techniques
•Techniques
•Formant Based •Concatenative HMM Based
•Rule Based
•Difficult to make
•Machine Like
•Instance Based
•Based on corpus
•Better quality
•Not flexible
•Statistical Based
•Based on corpus
•Newest technique
•More flexible
6 HMM Based Speech Synthesis
Agenda
Speech SynthesisHMM Based Speech SynthesisProposed SystemChallenges
7 HMM Based Speech Synthesis
HMM Based Speech Synthesis Overview
HMM has been used successfully in speech recognition.
In Recogntion
In Speech Synthesis:
)|(maxarg* OPOO
)|(maxarg*
OP
8 HMM Based Speech Synthesis
HMM Based Speech Synthesis Overview, Cont. Include delta and acceleration to get smooth
output
9 HMM Based Speech Synthesis
The Overall System
Synthesis Part
Speech Database F0
ExtractionMel-Cepstral
Analysis
HMM Training
Models
Labels and context features
Text Analysis
Text
Text Analysis Parameters Generation
Labels and context features
Pulse or Noise Excitation
f0
MLSA filter Speech
Mel-cepstrum
Excitation
Mel-cepstrum
f0
Training Part
10 HMM Based Speech Synthesis
The Overall System
Synthesis Part
Speech Database F0
ExtractionMel-Cepstral
Analysis
HMM Training
Models
Labels and context features
Text Analysis
Text
Text Analysis Parameters Generation
Labels and context features
Pulse or Noise Excitation
f0
MLSA filter Speech
Mel-cepstrum
Excitation
Mel-cepstrum
f0
Training Part
Modeled using MSD-HMM 25 Mel-Cepstral
11 HMM Based Speech Synthesis
The Overall System
Synthesis Part
Speech Database F0
ExtractionMel-Cepstral
Analysis
HMM Training
Models
Labels and context features
Text Analysis
Text
Text Analysis Parameters Generation
Labels and context features
Pulse or Noise Excitation
f0
MLSA filter Speech
Mel-cepstrum
Excitation
Mel-cepstrum
f0
Training Part
Context Dependant Models
Each model 5 States
12 HMM Based Speech Synthesis
The Overall System
Synthesis Part
Speech Database F0
ExtractionMel-Cepstral
Analysis
HMM Training
Models
Labels and context features
Text Analysis
Text
Text Analysis Parameters Generation
Labels and context features
Pulse or Noise Excitation
f0
MLSA filter Speech
Mel-cepstrum
Excitation
Mel-cepstrum
f0
Training Part
13 HMM Based Speech Synthesis
The Overall System
Synthesis Part
Speech Database F0
ExtractionMel-Cepstral
Analysis
HMM Training
Models
Labels and context features
Text Analysis
Text
Text Analysis Parameters Generation
Labels and context features
Pulse or Noise Excitation
f0
MLSA filter Speech
Mel-cepstrum
Excitation
Mel-cepstrum
f0
Training Part
Each Frame is either voicedor unvoiced
14 HMM Based Speech Synthesis
The Overall System
Synthesis Part
Speech Database F0
ExtractionMel-Cepstral
Analysis
HMM Training
Models
Labels and context features
Text Analysis
Text
Text Analysis Parameters Generation
Labels and context features
Pulse or Noise Excitation
f0
MLSA filter Speech
Mel-cepstrum
Excitation
Mel-cepstrum
f0
Training Part
15 HMM Based Speech Synthesis
Advantages
1. Its voice characteristics can be easily modified,
2. It can be applied to various languages with little modification,
3. A variety of speaking styles or emotional speech can be synthesized using the small amount of speech data,
4. Techniques developed in ASR can be easily applied,
5. Its footprint is relatively small. An HMM based TTS system produced best
results in Blizzard challenge.
16 HMM Based Speech Synthesis
Agenda
Speech SynthesisHMM Based Speech SynthesisProposed SystemChallenges
17 HMM Based Speech Synthesis
Problems we tried to solve
1. Marking each frame as either voiced or unvoiced degrades quality, because there are some unvoiced components on most voiced speech parts, and there are mixed-excitation phonemes.
2. Used speech signal analysis / synthesis techniques and parameters degrades quality.
18 HMM Based Speech Synthesis
Multi-Band Excitation
In MBE (Multi-Band Excitation) speech is divided into a number of frequency bands, and voicing is estimated in each band (used 17 bands).
19 HMM Based Speech Synthesis
Mixed Excitation
In synthesis periodic and noise excitations are mixed according to voicing parameters
20 HMM Based Speech Synthesis
Spectral Envelop Estimation
Find values for a fixed number of samples
Use sinusoidal model for synthesis
21 HMM Based Speech Synthesis
Modified System
Synthesis Part
Speech Database F0
ExtractionSpectral Envelop
Analysis
HMM Training
Models
Labels and context features
Text Analysis
Text
Text Analysis Parameters Generation
Labels and context features
Spectral Envelop Samples
f0
Training PartBands Voicing
detectionBands Voicing
Noise + STFT filter
HarmonicsSynthesis
Bands Mixing
Spec. Env. Samples+ f0
Bands Voicing
Voiced Speech
Unvoiced Speech Speech
22 HMM Based Speech Synthesis
Result
MOS scores
1
1.5
2
2.5
3
3.5
4
4.5
5
BaselineSystem
Baseline +MBE
ProposedSystem
Sc
ore
23 HMM Based Speech Synthesis
Agenda
Speech SynthesisHMM Based Speech SynthesisProposed SystemChallenges
24 HMM Based Speech Synthesis
Other Challenges
Speech is overly smoothed– Use global variance.
Modeling accuracy, the system uses same modeling as recognition.
– Hidden semi markov models (duration).
– Trajectory HMMs,
– Minimum Generation error training
– More states clusters and use acoustic context (under research).
25 HMM Based Speech Synthesis
More States Clusters
Instead of computing one Gaussian per state, we store all occurrences. And record the context of each occurrence.
At synthesis we get the best sequence using dynamic programming.
Previous NextCurrent
…
26 HMM Based Speech Synthesis
Thank You