Machine Learning in Spoken Language

Processing

Lecture 21

Spoken Language Processing

Prof. Andrew Rosenberg

2

Spoken Language Processing

• How machines can interact with speech.

• Speech Recognition• Speech Synthesis• Analysis of Speech

• Lexical Processing• Acoustic Signal Processing

3

Processing Tools

• In many applications, a common trajectory can be observed.

– Manually written rule-based systems

– Corpus-based Analysis

– Automatic training of systems via Machine Learning

4

Rule based systems

• Expert knowledge of a domain

• Often based on un-tested hypotheses

• Brittle– These are difficult to modify– Often have complex interdependencies– Rarely are able to determine

“confidence”

5

Machine Learning Systems

• Learn the relationship between– A Feature Vector, and – A dependent variable {label, or

number}

Classifier

Training data

Learning Algorithm

Classifier

HypothesisFeature Vector

Feature Vectors

Labels

6

Where do we use machine learning?

• The trajectory in natural language processing and speech has been from– manually written rules, to– automatically generated rules learned

from an abundance of data

• Speech Recognition• Speech Synthesis• Prosodic Analysis• Segmentation• Grapheme to

phoneme conversion• Speech act

classification• Disfluency

Identification• Emotion classification• Speech segmentation• Part of speech

tagging

• Parsing• Translation• Turn-taking• Information

Extraction

7

How do we use machine learning

• The Standard Approach to learning– Identify labeled training data– Decide what to label – syllables or words– Extract aggregate acoustic features

based on the labeling region– Train a supervised classifier– Evaluate using cross-validation or a

held-out test set.

8

What’s the role of linguistics?

• How do the rule based systems inform machine learning?

• Feature Representations.– The way we represent an entity or

phenomenon is informed by intuitions and prior study.

– The process of hand generating rules has moved to hand generation of Feature Extraction methods

9

What are the favorite tools in SLP?

• Decision Trees• Support Vector Machines

– Conditional Random Fields

• Neural Networks• Hidden Markov Model• k-means• k-nearest neighbors

• Graphical Models• Expectation maximization

10

Training, Development and Testing

• Available data is commonly divided into three sets– Training

• Used to train the model

– Development• Used to learn the best settings for

parameters

– Testing• Used to evaluate the performance of the

model trained on the training data with parameters l

11

Cross-Validation

• Cross Validation is a technique to estimate the generalization performance of a classifier.

• Identify n “folds” of the available data.• Train on n-1 folds• Test on the remaining fold.• Calculate average performance

…

12

Stratified Cross-validation

• Some classes have skewed distributions– For example, parts of speech.

• When creating cross validation folds, the class distribution is maintained across all folds

Function Noun Verb Adj.

13

Dimensionality

• In general, the more dimensions that a feature vector has, the more training data is necessary for reliable learning.– Some classifiers are more sensitive to

this than others.

• When we have a vocabulary of size N, this is often converted to N binary variables.

• This can quickly lead to an enormous feature space.

14

Dimensionality Reduction techniques

• Dimensionality reduction techniques are commonly used to reduce the number of dimensions, while keeping as much information as possible

• Regularization• Principle Components Analysis• Multi-dimensional scaling• Quantization

15

Next Time

• Working Session• Anonymous Course Feedback