seeing shakespeare - graphics.cs.wisc.edu · feng liu –multimedia,video supported bynsf,adobe)...

Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012

1

Seeing Shakespeare (and Sequences):

Making Pictures to understand things you don’t want to read

Michael GleicherDept of Computer SciencesUniversity of Wisconsin ‐Madison

Acknowledgements• All of this work is done in collaboration with a great group of students and collaborators

• This talk is work done with students:(who didn’t want their pictures shown)

Feng Liu – multimedia, video(work supported by NSF, Adobe)Now at Portland State Univ

Greg Cipriano– molecules, vis(work supported by DOE,NIH)Now at Solidworks

Aaron Bryden– molecule motion(work supported by NIH)Now at DOE Ames Lab

Danielle Albers – genomics(work supported by NSF, DOE)

Adrian Mayorga– vis(work supported by NIH)

Michael Correll – humanities, virology(work supported by NSF, NIH)

Kevin Ponto– VR(work supported by NIH)

Sean Andrist, Tomislav Pejsa– agents(work supported by NSF)

Zack Krejci – vis(volunteer)










Data Visualization

why Comp Sci Vis research might apply to you(especially if you are a humanist) Where is this guy coming from?

Some context

Some Stuff I Do(besides Visualization)

Analysis of Proteins Motion Synthesisfor Characters

Video Authoring

Image and VideoRetargeting

Non‐Verbal Cues forCommunicative Agents

Virtual Reality for Home Healthcare

What do these have in common?

It’s all stuff I’ve done in the past few years

It involves large amounts of data

It involves creating effective presentations

It requires some understanding of the datain order to simplify it







2






Does this all tie together?

How can we use our understanding of human perception and artistic traditions to improve our tools for communicating and comprehending?



Talk Roadmap

Molecular Surface Abstraction

Explanations for Exploration Literature without Reading

Molecular Motions

Sequence Comparison

Talk Roadmap



Molecular Motions

Sequence Comparison

Biological Applications

BiologyTo

Humanities

Humanities Applications

Talk Roadmap



Molecular Motions

Sequence Comparison

ArtAnd

Perception

Art

Perception/Design

Application

The Future:Computational

Thinking


3


Molecular Motions

Sequence Comparison


Prelude: Art or Perception? A Protein Surface

Work with Greg Cipriano and George Phillips

An aside…How do scientists look at proteins?

Stick and Ball Model (internals)



Ribbon Diagram (internals)



Molecular Surface (externals)

A Protein Surface


Work with Greg Cipriano and George Phillips. TVCG 2007, NAR 2010.

What’s Happening?

Simplification

Stylized Display

Surface Indications

Art

Abstraction

Good Lighting

Line Drawings

Non‐Photorealism

Visual Cognitive Science

Cue reduction

Provide Depth Cues

Enhance Contours

Tolerance of Shading


4

Why does B fight cancer?


Molecular Motions

Sequence Comparison


Inspired by Art (and a math trick) …

From Cutting (taken from McCloud), Representing motion in a static image, 2002

Molecular Motions

Bryden, Phillips, Gleicher. TVCG Jan ‘12

Molecular Motions

Coarse‐grained models

Normal‐mode Analysis (NMA)

Motion Illustration Motion Illustration

Artistic Inspirations:Comic Books / Diagrams

Abstract

Model

Illustrate


5

The Problem:Vector per point

AbstractGroup to fit models

Model

Illustrate


ModelAffine model per group

Illustrate x = A x

x = A x


ModelAffine model per group

IllustrateAffine ExponentialsGlyph Design

x = A x

x = A x

p(t) = eAt

p(t) = eAt

x = A x

x = A x

p(t) = eAt

p(t) = eAt


Molecular MotionsMolecular Surface Abstraction

Learning from perception…

Sequence Comparison


6

How do we interpret massive amounts of sequence data?

A tough problem…

TACTAGCTAGTAGCTAGCATCGACTACGACTGAC TACTAGCTAGTAGCTAGCATCGACTACGACTGAC

GCTAGCTGTTGCTAGCCGACTCATCGACTAAGCT

TCGACTAGCTAGATCGACTTATCGACTCACACTA

CTGGCTAGTTACACTATCTACCGACTGATCGACT

TACTAGCTAGTAGCTAGCATCGACTACGACTGAC

























7

Mauve sequence alignment and visualization. Perna Lab.

How to see anything in this sea of data?

How to see anything in this sea of data?

1. Be Realistic2. New Designs based on Perception

Scalable Overviews of Whole Genome Sequence Alignments

Sequence Surveyor

Albers, Dewey, Gleicher. IEEE TVCG (InfoVis 2011)

Number of Genomes Length of Genomes Types of Inquiry

Scalable


8

Mauve (Perna lab) Sequence Surveyor

Visual Search

Visual Clutter Summarization

Pre-Attentive Phenomena

Perceptual Principles

Visual Search



Visual Search


Pre-Attentive Phenomena Visual Search


Pre-Attentive Phenomena Visual Search



What are you searching for?

What textures form? What statistics do you get?

What pops out?

What are you looking for? MappingColor Mapping Color Schemes Position Mapping

Index Membership Freq Grouped Freq Pos in Reference

Index

Group

ed Freq

Pos in Re

ference

Combinations of different color and position mappings reveal interesting things in the data


9

Blocking

Group (relatively) continuous sets of neighboring genes into a single unit

roftilS yaeQ phnA tadG

Aggregate Encodings

Average

Aggregate Encodings

Average Robust Average

Color Weaving Event Striping

Anecdotes: 100 Bacteria

Conservation relationships between different families of genomes

Color by position in reference (arrow), order by relative ordering

Anecdotes: Buchnera

Color by position in reference (arrow), order by set of genomes containing each gene

Anecdotes: Buchnera

Averaging:

No significant trend

Color Weaving: Overall distribution

Anecdotes: Fungi

Bioinformatics applications allow users to test algorithms using visual checks

Color by overall frequency, order by relative ordering

Anecdotes: Fungi

Bioinformatics applications allow users to test algorithms using visual checks

Color by position in a reference, order by relative ordering

Another ApplicationSince I can’t pronounce the Fungi …


10

A Different Kind of Evolution . . .

Google Books Word Count DataDe

cade

s ‐>

Word Rank ‐> Color by position in reference

1600: and the of to a in that it is not ...

1610: the of and to in a that his for is ...

1620: the and of to in that a is he his ...

1630: the of and in to a was that his as ...

1640: the of and to in a that was is it ...

1650: the of and to a in is that as it ...

1660: the and of to in a that is it he ...

. . .

1970: the of and to in a is that for as ...



2000: the of and to in a that is for was ...


Google Books Word Count Data

Decade

s ‐>

Word Rank ‐> Color by position in reference


Google Books Word Count Data

Decade

s ‐>

Word Rank ‐> Color by occurrence count

Number of Decades (aggregate)

Event Striping Selection: 15 Decades


11

Does this really work?Color Weaving for Aggregate Displays

Correll, Albers, Franconeri, Gleicher. Comparing Averages in Time Series Data. CHI 2012.


12

A Conversation aboutVisualization


The spectra of Visualization

“Information”Visualization

“Scientific”Visualization

The spectra of Visualization

PresentationVisualization

ExploratoryVisualization

Science(of Visualization)

Practice(of Visualization)

Art(of Visualization)

Science(of Visualization)

Practice(of Visualization)

Art(of Visualization)

Caveat:Domain Science vs. Visualization Science

&

Tool Users

Toolsmiths

Theoreticians Designers

What do you need to know?

Domain science

Art / DesignPerception (Visual Cognition)Implementation (graphics, stats, databases, …)

Examples of other things that worked

The Class 2010Voluntary OverloadScheduled at Last MinuteGrad Special Topics (838)

14 “paying customers”4 dissertators/staff / facultyLess than half from CS

Can I learn this?

2012Regularly Scheduled ClassAdvertised via friendsUndergrad / Grad “meets with”

28 (+1) paying customers20 departments in original roster11 on final

Can I teach this?


13

http://graphics.cs.wisc.edu/Courses/Visualization12/

Case Study

What is VisualizationWhy VisualizeHow to Evaluate

Perception 101EncodingsColor

Multi‐VariateScalabilityInteraction

Case Study

Graphs / Networks / TreesAnimation / Presentation3D

2010Voluntary OverloadScheduled at Last MinuteGrad Special Topics (838)

14 “paying customers”4 dissertators/staff / facultyLess than half from CS

Can I learn this?

2012Scheduled special topicsAdvertised via friendsUndergrad / Grad “meets with”

28 (+1) paying customers20 departments in original11 departments on final

Can I teach this?

201XCreate a new courseHow to spread the word?What level?

Who to teach?What to teach them?Diversity of Needs

Should I teach this?Who to teach to?

Basic Math <‐> Statistics Grad Students

Artistically Inexperienced <‐> (ex‐) Pro Designer

Never Programmed <‐> Expert Implementer

No pet problems <‐> Has a “home” domain

Sophomore <‐> Dissertator

Never thought about it <‐> Visually Literate

Writes like an Engineer/Scientist <‐> Writes like a Huamities/Social Scientist

Who is interested?

A Research Program?

Lots of fun domains!

Visual ComparisonsA Common Thread?

There are general principles that apply across domains, data types, …

Visual Comparison:

And if we can figure it out, it’ll be easier to crank out the comparison tools/techniques quickly

Mike’s theory of visual comparison 0.2*

* This is a work in progress.Comments welcomed!


14

The 3 3s

3 Axes of Scalability / Hardness3 Strategies for Scalability3 Basic Designs

Comparison is easy. Until it gets hard.

3 Axes of Hardness

Number of things to compareSize/complexity of things to compareComplexity of the relationships

People can only compare a few things at a time*3 Strategies for scalingSelect SubsetSummarize StatisticallyScan Serially

My cognitive scientist friends say the magic number 7+/‐2 is an over‐simplification, but …

All designs appear to fall into 3 categories

3 Basic Designs*

* Each has its pros and cons

Develop Methods Specifically to Support Visual Comparison

Visual Comparison Punchline:

One example…

Mayorga and Gleicher. Splatterplots: Overcoming Overdraw. Submitted for publication.

Scatterplot? Splatterplot!

Mayorga and Gleicher. Splatterplots: Overcoming Overdraw. Submitted for publication.


15

Goo

gle Bo

oks

(broad

set)

Novels

Sequence Comparison

Explanations for Exploration


Humanities Scholarship

Literature without Reading

Study Literature without Reading?

See patterns across language

Consider Larger Collections of Books

See small scale patterns in familiar texts

Be uncultured and still hang out with the cool kids

1. Do measurements of texts2. Make inferences from the statistics3. Build humanist‐style arguments


16

Until we take the time to learn about how the other side thinks, we can’t really work together.

Once we learn how each other thinks, our ways of thinking can infuse each other’s.

This is not just building tools for our friends.It’s a lotmore fun and interesting

The statistics are not the argument

Exemplars and OutliersGo back to the sources

Arguments based on context and knowledge

Multiple viewpoints and lenses

“Humanist” Collaborators:Robin Valenza*, Mike Witmore, Jonathan HopeCathy DeRose, Jason Whitt, …

Comp Sci Collaborators:Michael Correll, Danielle Albers, Zack Krejci, …

* Robin had a prior life in which she was one of us

1. Do measurements of texts2. Make inferences from the statistics3. Build humanist‐style arguments

1: Count Text tagging

Text Vector

4, 0, 3.6, 4.7, 0, 3.4, …

Count of wordCategory 1



CouCate

Texts Vectors

4, 0, 3.6, 4.7, 0, 3.4, …3, 2.4, 0, 4.2, 4.7, 5, …1.5, 2.3, 0, 1.2, 6.2, ……

Pieces of Texts Vectors

4, 0, 3.6, 4.7, 0, 3.4, …3, 2.4, 0, 4.2, 4.7, 5, …1.5, 2.3, 0, 1.2, 6.2, ……

Just counting

Words (phrases) have a type (tag)

DocuscopeSimple matching against a dictionaryHand‐built dictionaries

100‐115 Categories, 12‐17 Clusters (groups)


17

N

Not in dictionary

Not Tagged

15 “Clusters” (115 LATs)

How to look at 100+ dimensions?

Visualize them directlyDimensionality Reduction

Let the machine do it for you

1. Make Picture2. Read tea leaves

How to support “Humanist” arguments?

The statistics are not the argument

Exemplars and OutliersGo back to the sources

Arguments based on context and knowledge

Multiple viewpoints and lenses


18

Domain Specific Tools

Correll, Witmore, Gleicher. Exploring Collections of Tagged Text for Literary Scholarship.Computer Graphics Forum (Proceedings Eurovis), 2011.

Back to the details that cause patterns


Sequence Comparison


And for my next trick…


Exploring High‐Dimensional Spaces

Gleicher. Unpublished Work in Progress

What we have:Measurements (counts) of things

What we want:Explanations of how these lead to properties of the objects that we care about

Shakespeare’s PlaysA Source of Examples

Genre?

AutoBio

Upd

ates


19

Request

Time Shift

Positivity

Negativity

The Comedicness axis?

Puts comedies on the rightNon‐comedies on the left

Provides a “measurement” of comedicness

Just a different view of the data

Positivity

Negativity

Dividing Line(best I can do)

Comedicness0 1 2 3‐1

Comedicness

Tragicne

ss

Comedicness

Tragicne

ss


20

Using expert knowledge to guide exploration

Is this cheating?

Machine LearningCorrectness

GeneralizesLarge Margin (gap)Efficient, Robust, Concise(proxies for generality)

Classifiers

Explanation BuildingCorrect – or interesting

Leads to good argumentsSimple (concise)Aligns with other knowledgeProvides multiple viewpointsProvides exemplarsOutliers to examine

Explainers!

-2.702 Immediacy - 2.519 Negativity + 1.894 Positivity - 1.846 ReportingStates + 1.160 FirstPer + 0.990 DirectAddress + 0.536 Question -0.395 CommonAuthorities -0.164 ProjectAhead + 0.067 ReportingEvents

(all 36 correct, large margin, 10 variables)

-0.304 Negativity + 0.147 FirstPer -0.095 CommonAuthorities

(5 “wrong”, 3 variables)

-10 Negativity + 5 FirstPer - 3 CommonAuthorities


-2 Negativity + 1 FirstPer - 1 CommonAuthorities


Explanations from Expert Explorations

Generate Explanations from given relationsTrade off accuracy vs. parsimony

Add additional knowledge to shape viewsGenerate alternate explanations

No additional ConstraintsSimpler ‐

Hamlet and TwelfthNight ConstrainedSimpler ‐

z =SVMpursuit(d,cp,5,svmparamsched=[.1,.2,.3,.4],quantize=5)

-5 Negativity + 2 FirstPer - 2 ComAuthor(3 variables)

-5 Inclusive + 3 Positivity - 2 AbstractConcepts + 1 PersProp(4 variables)

5 SelfDisclosure - 4 Motions - 2 StandardsPos + 1 DirectAddress(4 variables)

5 PredFuture - 2 SpaceRelation -2 Resistance + 1 Question + 1 PersProp - 1 SenseObject

(6 variables)

All with 32 correct


21

What do you want to ask?How do you want to look at data?

Expert Explorations lead to Explanations

Can you identify the genre of an act?Just based only on word usage…

We’re telling it what to explain

Stuff it “discovers” helps confirm

What act the play is?Can the word usage tell you…

I

II

III

IV

V

Presentation and Interactionis Challenging!

Isn’t this a Visualization talk?


22

Uniqueness of a play?One more Shakespeare case study

A Midsummer Night’s Dream

35 Other Plays

Person Property

Sense Object

Still need …

an integrated system for experimenting

ways to specify knowledgeways to assess explanationssystematic ways to generate alternatives

to see if this works in more casesto expand to richer classes of explanations

to do a lot of work. . . . .

Until we take the time to learn about how the other side thinks, we can’t really work together.

Once we learn how each other thinks, our ways of thinking can infuse each other’s.

This is not just building tools for our friends.It’s a lotmore fun and interesting


Sequence Comparison


Where have we been?


Talk Roadmap



Molecular Motions

Sequence Comparison

Biological Applications

BiologyTo

Humanities

Humanities Applications

Talk Roadmap



Molecular Motions

Sequence Comparison

ArtAnd

Perception

Art

Perception/Design

Application

The Future:Computational

Thinking


Thanks!To you for listeningTo Robin and Carrie for inviting me

To my students and collaborators

To the folks who pay the bills(NSF, NIH, Mellon, …)

Michael [email protected] of Computer SciencesUniversity of Wisconsin ‐Madison

seeing shakespeare - graphics.cs.wisc.edu · feng liu –multimedia,video supported bynsf,adobe)...

Documents