Interpretability

Advanced Machine Learning for NLPJordan Boyd-GraberNEED FOR INTERPRETABILITY

Slides adapted from Marco Tulio Ribeiro

Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 1 of 1

Classification

• But representation is often less important (means to end)

• We really care about end result

• And not doing simple things like decision trees / linear classifier

• That’s why we’re making complicated algorithms

CanexplainthismessJ

Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 2 of 1

Classification

• But representation is often less important (means to end)

• We really care about end result

• And not doing simple things like decision trees / linear classifier

• That’s why we’re making complicated algorithms

CanexplainthismessJ

Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 2 of 1

LIME

16

Locally-faithfulsimpledecisionboundary

èGoodexplana:onforpredic:on

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. “Why Should ITrust You?” Explaining the Predictions of Any Classifier.LIME: Local Interpretable Model-Agnostic Explanations

Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 3 of 1

What’s an Explanation

Whydidthishappen?HowdoIfixit?

Appearin21%oftrainingexamples,almostalwaysinatheism

Appearsin11%oftrainingexamples,alwaysinatheism

11

From: Keith Richards Subject: Christianity is the answer NTTP-Posting-Host: x.x.com I think Christianity is the one true religion. If you’d like to know more, send me a note

Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 4 of 1

What’s an Explanation

P()=0.21P()=0.24P()=0.32

Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 4 of 1

What makes good Explanation?

• Interpretable: Humans can Understand

• Faithful: Describes Model

• Model Agnostic: Generalize to Many Models

Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 5 of 1

Method

• Sample points around xi

• Use complex model to predict labels for each sample• Weigh samples according to distance to xi

• Learn new simple model on weighted samples• Use simple model to explain

Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 6 of 1

Perturbing an Example

x'(con(guoussuperpixels)x(3colorchannels/pixel)

Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 7 of 1

Perturbing an Example

x(embeddings)

0.5 0.3 1.3 4.4 1.1 ...

x'(words)

Thisisahorriblemovie.

Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 7 of 1

Image Example

PerturbedInstances P(Labrador)

OriginalImage

20

0.92

0.001

0.34

P(labrador)=0.21

Locallyweightedregression

ExplanaDon

Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 8 of 1

Is this a good Classifier?

Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 9 of 1

Is this a good Classifier?

0102030405060708090

100

Didn'ttrustthemodel

"Snowinsight"

%ofsub

jects(ou

tof2

7)

BeforeexplanaIons AKerexplanaIons

Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 9 of 1

Improving ML Algorithms

20newsgroups-Train

20newsgroups-test

Hiddenreligiondataset

train predict

Explain

REPEAT

Evaluate

Turkersdon’tknowaboutthisdataset

Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 10 of 1

Improving ML Algorithms

Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 10 of 1

Improving ML Algorithms

0,5

0,55

0,6

0,65

0,7

0,75

0,8

Nouserinput 1round 2rounds 3rounds

Acc

urac

y on

hid

den

set

Trainon20newsgroups

Trainonhand-cleaned20newsgroups

Trainon20newsgroupsturkerscleandata

Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 10 of 1