and away we go: understanding the complexity of launching ...jbg/docs/itm_pres.pdf · nasa shuttle...
TRANSCRIPT
![Page 1: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/1.jpg)
Yuening Hu, Jordan Boyd-‐Graber, Brianna Satinoff{ynhu, bsonrisa}@cs.umd.edu, [email protected]
University of MarylandJune 20, 2011
Interactive Topic Modeling
The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies
![Page 2: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/2.jpg)
Introduction of Topic ModelsDiagnosing Topic ModelsEncoding Feedback to Topic ModelsStrategiesExperimentsConclusionFuture Steps
Outline
2
![Page 3: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/3.jpg)
Introduction of Topic ModelsDiagnosing Topic ModelsEncoding Feedback to Topic ModelsStrategiesExperimentsConclusionFuture Steps
Outline
3
![Page 4: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/4.jpg)
Why topic models?
A huge number of documentsWant
4
![Page 5: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/5.jpg)
Why topic models?
A huge number of documentsWant
Topic ModelsA corpus-level view of major themesUnsupervised
5
![Page 6: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/6.jpg)
Conceptual approach
What topics are expressed throughout the corpusWhat topics are expressed by each document
6
TOPIC 1
computer, site, technology, system, service, phone, internet, machineTOPIC 2
Sell, sale, market, product, business, advertising, storeTOPIC 3
play, film, movie,theater, production, star, director, stage
![Page 7: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/7.jpg)
A generative probabilistic model of documents that posits a hidden topic structureLatent Dirichlet Allocation (LDA) (Blei et al., 2003)
A topic is a distribution over wordsA document is a distribution over topics
7
![Page 8: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/8.jpg)
8
Measure topic quality (Chang et al., 2009), not all topics are goodIt is easy to be detected by humans
![Page 9: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/9.jpg)
Measure topic quality (Chang et al., 2009), not all topics are goodIt is easy to be detected by humans
Good Topicartist
exhibitiongallerymuseumpainting
Bad Topiccommitteelegislationproposalrepublictaxis
9
![Page 10: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/10.jpg)
Introduction of Topic ModelsDiagnosing Topic ModelsEncoding Feedback to Topic ModelsStrategiesExperimentsConclusionFuture Steps
Outline
10
![Page 11: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/11.jpg)
Diagnosing topic models
Topic 1 Topic 2shuttle NASAlaunch telescoperacket quasar
battledore saturnbackhand spaceastronaut moon
11
![Page 12: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/12.jpg)
Diagnosing topic models
Topic 1 Topic 2shuttle NASAlaunch telescoperacket quasar
battledore saturnbackhand spaceastronaut moon
shuttle, launch andNASA should be together.
12
![Page 13: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/13.jpg)
Diagnosing topic models
Topic 3bladder
spinal_cordsci
spinalurinary
urothelialcervical
urinary_tractlumbar
13
![Page 14: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/14.jpg)
Diagnosing topic models
belong together!Should be separated.
Topic 3bladder
spinal_cordsci
spinalurinary
urothelialcervical
urinary_tractlumbar
14
![Page 15: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/15.jpg)
Simple interaction
Topic Models
Topic 1
sampling
check
feedback
15
![Page 16: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/16.jpg)
Introduction of Topic ModelsDiagnosing Topic ModelsEncoding Feedback to Topic ModelsStrategiesExperimentsConclusionFuture Steps
Outline
16
![Page 17: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/17.jpg)
What feedback?
Topics are distributions over uncorrelated words
17
![Page 18: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/18.jpg)
What feedback?
Topics are distributions over uncorrelated wordsAdd Constraints: positive and negative correlations
18
![Page 19: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/19.jpg)
Prior in normal LDA
Same prior for all the words (Boyd-‐Graber et al., 2007)
nasa shuttle space tea bagel god constitution president
19
![Page 20: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/20.jpg)
Model constraints as prior
Dirichlet Forest: prior tree structure(Andrzejewski et al. 2009)Positive constraints only in this paper
nasa shuttle space tea bagel god constitution president
nasa shuttle space
tea bagel god
constitution president
20
C1 C2
C1 C2
![Page 21: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/21.jpg)
How to incorporate feedback?
Topic Models with a prior structure
Topic 1
Incrementallytopic learning
Get feedback from users
Build prior structure
Start with same prior
21
![Page 22: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/22.jpg)
How to incorporate feedback?
Topic Models with a prior structure
Topic 1
Incrementallytopic learning
Get feedback from users
Build prior structure
Start with same prior
22
How about the old topic assignment of each word?
![Page 23: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/23.jpg)
Introduction of Topic ModelsDiagnosing Topic ModelsEncoding Feedback to Topic ModelsStrategiesExperimentsConclusionFuture Steps
Outline
23
![Page 24: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/24.jpg)
Remember or forget?
Four strategiesAllNoneDocTerm
Toy example
24
![Page 25: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/25.jpg)
Toy example
Doc 1
nasa shuttle launch…
Doc 2
racket serve shuttle …
Doc 3
bladder pain bladder …
Doc 4
spine pain lumbar …
25
![Page 26: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/26.jpg)
Toy example: All
Doc 1
nasa shuttle launch…
Doc 2
racket serve shuttle …
Doc 3
bladder pain bladder …
Doc 4
spine pain lumbar …
+
Strategy AllForget all topic assignmentsStart from the very beginning
26
![Page 27: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/27.jpg)
Toy example: None
Doc 1
nasa shuttle launch…
Doc 2
racket serve shuttle …
Doc 3
bladder pain bladder …
Doc 4
spine pain lumbar …
Strategy NoneRemember everything Continue
27
![Page 28: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/28.jpg)
Toy example: Doc
Doc 1
nasa shuttle launch…
Doc 2
racket serve shuttle …
Doc 3
bladder pain bladder …
Doc 4
spine pain lumbar …
Round 1Positive constr: (nasa - shuttle)Strategy: Doc
28
![Page 29: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/29.jpg)
Toy example: Doc
Doc 1
nasa shuttle launch…
Doc 2
racket serve shuttle …
Doc 3
bladder pain bladder …
Doc 4
spine pain lumbar …
29
Strategy DocForget the topic assignments for docs containing constraintsRemember the otherscontinue
![Page 30: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/30.jpg)
Toy example: Doc
Doc 1
nasa shuttle launch…
Doc 2
racket serve shuttle …
Doc 3
bladder pain bladder …
Doc 4
spine pain lumbar …
Round 1Positive constr: (nasa - shuttle)Strategy: Doc
30
![Page 31: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/31.jpg)
Toy example: Doc
Doc 1
nasa shuttle launch…
Doc 2
racket serve shuttle …
Doc 3
bladder pain bladder …
Doc 4
spine pain lumbar …
Round 1Positive constr: (nasa - shuttle)Strategy: Doc
31
![Page 32: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/32.jpg)
Toy example: Doc
Doc 1
nasa shuttle launch…
Doc 2
racket serve shuttle …
Doc 3
bladder pain bladder …
Doc 4
spine pain lumbar …
Round 1Positive constr: (nasa - shuttle)Strategy: Doc
32
![Page 33: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/33.jpg)
Toy example: Doc
Doc 1
nasa shuttle launch…
Doc 2
racket serve shuttle …
Doc 3
bladder pain bladder …
Doc 4
spine pain lumbar …
Round 1Positive constr: (nasa - shuttle)Strategy: Doc
33
![Page 34: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/34.jpg)
Toy example: Doc
Doc 1
nasa shuttle launch…
Doc 2
racket serve shuttle …
Doc 3
bladder pain bladder …
Doc 4
spine pain lumbar …
Round 1Positive constr: (nasa - shuttle)Strategy: Doc
34
![Page 35: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/35.jpg)
Toy example: Doc
Doc 1
nasa shuttle launch…
Doc 2
racket serve shuttle …
Doc 3
bladder pain bladder …
Doc 4
spine pain lumbar …
Doc 1
nasa shuttle launch…
Doc 2
racket serve shuttle …
Doc 3
bladder pain bladder …
Doc 4
spine pain lumbar …
35
![Page 36: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/36.jpg)
Toy example: Term
Doc 1
nasa shuttle launch…
Doc 2
racket serve shuttle …
Doc 3
bladder pain bladder …
Doc 4
spine pain lumbar …
Round 2Negative constr: (spine - bladder)Strategy: Term
36
![Page 37: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/37.jpg)
Toy example: Term
Doc 1
nasa shuttle launch…
Doc 2
racket serve shuttle …
Doc 3
bladder pain bladder …
Doc 4
spine pain lumbar …
37
Strategy TermForget the topic assignments for the constraint words, Remember the othersContinue
![Page 38: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/38.jpg)
Toy example: Term
Doc 1
nasa shuttle launch…
Doc 2
racket serve shuttle …
Doc 3
bladder pain bladder …
Doc 4
spine pain lumbar …
Round 2Negative constr: (spine - bladder)Strategy: Term
38
![Page 39: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/39.jpg)
Toy example: Term
Doc 1
nasa shuttle launch…
Doc 2
racket serve shuttle …
Doc 3
bladder pain bladder …
Doc 4
spine pain lumbar …
Round 2Negative constr: (spine - bladder)Strategy: Term
39
![Page 40: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/40.jpg)
Toy example: Term
Doc 1
nasa shuttle launch…
Doc 2
racket serve shuttle …
Doc 3
bladder pain bladder …
Doc 4
spine pain lumbar …
Round 2Negative constr: (spine - bladder)Strategy: Term
40
![Page 41: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/41.jpg)
Toy example
Doc 1
nasa shuttle launch…
Doc 2
racket serve shuttle …
Doc 3
bladder pain bladder …
Doc 4
spine pain lumbar …
Doc 1
nasa shuttle launch…
Doc 2
racket serve shuttle …
Doc 3
bladder pain bladder …
Doc 4
spine pain lumbar …
41
![Page 42: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/42.jpg)
Introduction of Topic ModelsDiagnosing Topic ModelsEncoding Feedback to Topic ModelsStrategiesExperimentsConclusionFuture Steps
Outline
42
![Page 43: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/43.jpg)
Motivating exampleTopic Before
1 election, yeltsin, russian, political, party, democratic, russia, president, democracy, boris, country, south, years, month,government, vote, since, leader, presidential, military
2 new, york, city, state, mayor, budget, giuliani, council, cuomo, gov, plan, year, rudolph, dinkins, lead, need, governor, legislature, pataki, David
3 nuclear, arms, weapon, defense, treaty, missile, world, unite, yet, soviet, lead, secretary, would, control, korea, intelligence, test, nation, country, testing
4 president, bush, administration, clinton, american, force, reagan, war, unite, lead, economic, iraq, congress, america, iraqi, policy, aid, international, military, see
20 soviet, lead, gorbachev, union, west, mikhail, reform, change, europe, leaders, poland, communist, know, old, right, human, washington, western, bring, party
43
![Page 44: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/44.jpg)
Motivating exampleTopic Before
1 election, yeltsin, russian, political, party, democratic, russia, president, military, democracy, boris, country, south, years, month,government, vote, since, leader, presidential
20 soviet, lead, gorbachev, union, west, mikhail, reform, change, europe, leaders, poland, communist, know, old, right, human, ashington, western, bring, party
44
![Page 45: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/45.jpg)
Motivating exampleTopic Before
1 election, yeltsin, russian, political, party, democratic, russia, president, military, democracy, boris, country, south, years, month,government, vote, since, leader, presidential
20 soviet, lead, gorbachev, union, west, mikhail, reform, change, europe, leaders, poland, communist, know, old, right, human, ashington, western, bring, party
45
Suggested constraintboris, communist, gorbachev,mikhail, russia, russian, soviet,union, yeltsin
![Page 46: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/46.jpg)
Motivating example
46
Topic After1 election, democratic, south,
country, president, party, africa, lead, even, democracy, leader,presidential, week, politics, minister, percent, voter, last, month, years
20 soviet, union, economic, reform, yeltsin, russian, lead, russia, gorbachev, leaders, west, president, boris, moscow, europe, poland, mikhail, relations, communist, power
Topic Before1 election, yeltsin, russian,
political, party, democratic, russia, president, military, democracy, boris, country, south, years, month,government, vote, since, leader, presidential
20 soviet, lead, gorbachev, union, west, mikhail, reform, change, europe, leaders, poland, communist, know, old, right, human, ashington, western, bring, party
![Page 47: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/47.jpg)
Motivating exampleTopic Before
2 new, york, city, state, mayor, budget, giuliani, council, cuomo, gov, plan, year, David, rudolph, dinkins, lead, need, governor, legislature, pataki
3 nuclear, arms, weapon, defense, treaty, missile, world, unite, yet, soviet, lead, would, control, korea, intelligence, test, nation, country, testing
4 president, bush, military, see, administration, clinton, american, force, reagan, war, unite, lead, economic, iraq, congress, america, iraqi, policy, aid, international,
47
Topic After
2 new, york, city, state, mayor, budget, council, giuliani, gov, cuomo, year, rudolph, dink-ins, legislature, plan, david, governor, pataki, need, cut
3 nuclear, arms, weapon, treaty, defense, war, missile, may, come, test, american, world, would, need, lead, get, join, yet, clinton, nation
4 president, administration, bush, clinton, war, unite, force, reagan, american, america, make, nation, military, iraq, iraqi, troops, international, country, yesterday, plan
![Page 48: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/48.jpg)
Simulating an interactive user
Dataset: 20 News groupsConstraints from feature selection on training data
soc.religion.christiansabbath20 classes: 20 constraint sets, 21 words per constraint set
Add them to the topic model as positive constraintsAdd one word per class each time, 21 rounds in total
Train classifier on training dataUse topic distribution of each doc as the feature
Measure classification error rate of test data
48
![Page 49: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/49.jpg)
Which strategy & how long to wait?
Facet: number of iterations added per roundStart with 100 iterationsNull: no constraints, comparable iters
49
![Page 50: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/50.jpg)
Put humans in the loop
50
![Page 51: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/51.jpg)
Put humans in the loop
51
![Page 52: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/52.jpg)
Put humans in the loop
Some constraints users createdInscrutable
better, people, right, take, thingsfbi, let, says
Collocationsjesus, christsolar, suneven, numberbook, list
Common instances (e.g. first names)Soft constraint: mac, windows
52
![Page 53: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/53.jpg)
Negative constraints
NIH data(700 topics)Negative constraint: bladder spinal_cord
53
Topic Before318 bladder, sci, spinal_cord,
spinal_cord_injury, spinal, urinary, urinary_tract, urothelial, injury, motor, recovery, reflex, cervical, urothelium, functional_recovery
Topic After318 sci, spinal_cord,
spinal_cord_injury, spinal, injury, recovery, motor, reflex, urothelial, injured, functional_recovery, plasticity, locomotor, cervical, locomotion
![Page 54: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/54.jpg)
Conclusion
An efficient way to refine and improve the topics discovered by topic modelsA paradigm for non-‐specialist consumers to refine models to better reflect their interests and needsCreating tools to do soWe need users!
54
![Page 55: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/55.jpg)
Future steps
Speed upSuggesting constraintsIncorporating other domain knowledgeIncorporating interaction to other models
55
![Page 56: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/56.jpg)
Yuening Hu, Jordan Boyd-‐Graber, Brianna Satinoff{ynhu, bsonrisa}@cs.umd.edu, [email protected]
University of MarylandJune 20, 2011
Thank you! Any questions?
The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies
![Page 57: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/57.jpg)
Constrained LDA
Sampling equation
number of times the unconstrained word appears in topic knumber of times any word of constraint appears in topic k
the number of times word appears in constraint in topic kvocabulary sizenumber of words in constraint
57
![Page 58: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/58.jpg)
Which strategy?
All Full: all constraints are known, comparable itersAll Initial: all constraints are known, 100 itersNull: no constraints, comparable iters
58
![Page 59: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/59.jpg)
Put humans in the loop
59
![Page 60: And Away We Go: Understanding The Complexity of Launching ...jbg/docs/itm_pres.pdf · nasa shuttle space tea bagel god constitution president ... government, vote, since, leader,](https://reader031.vdocuments.net/reader031/viewer/2022030423/5aab6d147f8b9a693f8be5a5/html5/thumbnails/60.jpg)
Reference1. David M. Blei, Andrew Ng, and Michael Jordan. 2003. Latent Dirichlet allocatio
n. Journal of Machine Learning Research, 3:993 1022.2. Jonathan Chang, Jordan Boyd-‐Graber, Chong Wang, Sean Gerrish, and David
M. Blei. 2009. Reading tea leaves: How humans interpret topic models. In Neural Information Processing Systems.
3. David Andrzejewski, Xiaojin Zhu, and Mark Craven. 2009. Incorporating domain knowledge into topic modeling via Dirichlet forest priors. In Proceedings of International Conference of Machine Learning.
4. Jordan Boyd-‐Graber, David M. Blei, and Xiaojin Zhu. 2007. A topic model for word sense disambiguation. In Proceedings of Emperical Methods in Natural Language Processing.
5. Jonathan Chang. 2010. Not-‐so-‐latent dirichlet allocation: Collapsed gibbs sampling using human judgments. In NAACL Workshop: Creating Speech and Language Data With Mechanical Turk.
60