FhG Bonn, 26.02.2004

Generating Multimedia Presentations

Generating Multimedia

Presentations

Stefano Bocconi

Visual material taken from the film Minority report, TM and © 2002 Twentieth Century Fox and Dreamworks

FhG Bonn, 26.02.2004

Information Systems

Multimedia and Human Computer Interaction Group

We are currently working on four aspects:

modeling of domain-dependent and domain-independent discourse to steer the presentation generation process,

investigate dependencies of the user and domain models in the generation process,

investigate characteristics of media types for presenting information to the user.

investigate to what extent graphic design knowledge can be included in the generation process

FhG Bonn, 26.02.2004

The Genre of Documentaries

A documentary purports to present factual information about the world

The viewer is led to think that the facts are filmed as they happened

Video Documentary

FhG Bonn, 26.02.2004

Categorical Documentaries

Conveys information in a simple fashion over a specific category (or more)

Examples are scientific documentaries (national geography)

The category and its subcategories provide a scheme for the narrative

Video Documentary

FhG Bonn, 26.02.2004

Rhetorical Documentaries

The goal is to persuade the audience to adopt an opinion

Arguments from the source

Subject-centered arguments

Viewer-centered arguments

Video Documentary

The River, Pare Lorentz (1936)

FhG Bonn, 26.02.2004

Interview with America

Subject: the opinion of American people after 9-11

Shooting: from 27-10-2001 to 01-11-2001 in Stamford (CT), New York (NY), Boston (MA) and Cleveland (OH)

Editing: approximately 3 months

Video Documentary

http://www.interviewwithamerica.com/

FhG Bonn, 26.02.2004

The problems

Only one final version: what do we show?

Who is pro and who is against? What are the numbers?

Biased versus objective trade-off: a choice must be made

Video Documentary

FhG Bonn, 26.02.2004

Providing Access to the Source

Annotate the material:

What question was asked

Who answered (social categories)

Where

Possible query: show me all the answers to question “what do you think of the war in Afghanistan” given by black men

Open Documentary

FhG Bonn, 26.02.2004

Rhetorical Tools

According to Aristotle, rhetoric is "the ability, in each particular case, to see the available means of persuasion”

Three main forms of rhetoric: Ethos, Logos, and Pathos

Open Documentary

FhG Bonn, 26.02.2004

Ethos

Ethos is appeal based on the character of the speaker. An ethos-driven document relies on the reputation of the author

Ethos is used in what Bordwell calls Arguments from the source

Open Documentary

FhG Bonn, 26.02.2004

Logos

Logos is appeal based on logic or reason. Documents distributed by companies or corporations are logos-driven. Scholarly documents are also often logos-driven

Logos is used in what Bordwell calls Subject-centered arguments

Open Documentary

FhG Bonn, 26.02.2004

Pathos

Pathos is appeal based on emotion. Advertisements tend to be pathos-driven

Emotions have a functional role as constraints on our decision making (Damasio)

Ethos is used in what Bordwell calls Viewer-centered arguments

Open Documentary

FhG Bonn, 26.02.2004

Example

“I am never a fan of military action, in the big picture I don’t think it is ever a good thing, but I think there are circumstances in which I certainly can’t think of a more effective way to counter this sort of thing, I suppose there is a point in which certain people play by certain rules and you have to go to their level, I do not think there is any way to resolve this conflict diplomatically”

Open Documentary

Subject is Controversial

The position is Pro or Against

FhG Bonn, 26.02.2004

Using the Logos

Concession: I do not like A, but B

A = war in Afghanistan (military action)

B = only solution reasoning

A is the only solution because

B0) A is cruel

B1) the attack was cruel

B2) the response has to be like the attack

Open Documentary

PRO

AGAINST

FhG Bonn, 26.02.2004

Using the Logos

Different rhetorical figures, in general enthymemes:

War is the best thing to do because it will solve the problem

Unique : only A is possible

Antithesis : only B and A possible and B not a solution

Optimal : A is the best solution

Unconditional : A is true

Evidence : (A if B) and B is true

The reasoning can be explicitly expressed in the interview or belong to the “common sense”

Open Documentary

FhG Bonn, 26.02.2004

Using the Pathos

How and what emotions can we use to persuade?

Cognitive structure of emotions (Ortony)

The expressions for happiness, sadness, disgust, surprise, anger and fear are culture-, gender- and age independent (Ekman & Friesen)

Open Documentary

PRO

AGAINST

FhG Bonn, 26.02.2004

Strategies

We saw some “local” strategies

More high level strategies are possible, like increasing / decreasing or hyperbole

To form a longer presentations, higher level strategies are needed

Open Documentary

Rhetoric

Pathos Logos

Replace Augment

FhG Bonn, 26.02.2004

Issues

Advantages

Reuse the reasoning of the humans, no need for a very complex system like Cyc

Disadvantages

Extensive annotation is needed

Need for a knowledge base

Validation

User studies?

Mimicking the reality

FhG Bonn, 26.02.2004

Previous Work

“Media Streams is a system for annotating, retrieving, repurposing, and automatically assembling digital video. It uses a stream-based, semantic representation of video content with an iconic visual language interface of hierarchically structured, composable, and searchable primitives.”

AUTEUR: The Application of Video Semantics and Theme Representation for Automated Film Editing

FhG Bonn, 26.02.2004

Related Work

`Terminal Time is a history "engine:" a machine which combines historical events, ideological rhetoric, familiar forms of TV documentary, consumer polls and artificial intelligence algorithms to create hybrid cinematic experiences for mass audiences that are different every single time’

FhG Bonn, 26.02.2004

References

Damasio, Antonio - Descartes' Error: Emotion, Reason, and the Human Brain, Avon Books, 1994.

Bordwell, D. (1989). Making Meaning - Inference and Rhetoric in the Interpretation of Cinema. Cambridge, Massachusetts: Harward University Press.

Davis, M. Media Streams: Representing Video for Retrieval and Repurposing, Massachusetts Institute of Technology, Cambridge, Massachusetts, 1995.

Nack, F. AUTEUR: The Application of Video Semantics and Theme Representation for Automated Film Editing, Lancaster University, 1996.

Ortony, A., Clore, G. L. and Collins, A. - The Cognitive Structure of Emotions. Cambridge University Press, 1988.

Ekman, P. and Friesen, W.V., Facial Action Coding System. Consulting Psychologists Press Inc., 1978.

Terminal Time http://www.terminaltime.com/.

FhG Bonn, 26.02.2004

Discussion