2005.03.10 SLIDE 1IS146 – SPRING 2005

Reading Visual Representations

Prof. Marc Davis, Prof. Peter Lyman, and danah boyd

UC Berkeley SIMS

Tuesday and Thursday 2:00 pm – 3:30 pm

Spring 2005http://www.sims.berkeley.edu/academics/courses/is146/s05/

IS146:

Foundations of New Media

2005.03.10 SLIDE 2IS146 – SPRING 2005

Lecture Overview

• Review of Last Time– Social Uses of Mobile Phones

• Today– Reading Visual Representations

• Preview of Next Time– Midterm Exam

2005.03.10 SLIDE 3IS146 – SPRING 2005

Lecture Overview

• Review of Last Time– Social Uses of Mobile Phones

• Today– Reading Visual Representations

• Preview of Next Time– Midterm Exam

2005.03.10 SLIDE 4IS146 – SPRING 2005

Why Does Texting Work In Japan?• Over 10 years since popular mobile texting

beginning with youth pager adoption• Dense and information-rich pedestrian urban

ecologies• Limited private space • Highly regulated public transportation• High print and visual literacy rates• Vibrant popular techno-cultures surrounding

portable electronics• Text the dominant modality of mobile

communicationSlide from Ito

2005.03.10 SLIDE 5IS146 – SPRING 2005

Diary Study

• What is a diary study?– Record every interaction with the mobile– Indicate meta commentary– Complements data record

• When are diary studies useful?– Interactions over extended periods of time– Want to know what people are thinking during

interaction before they forget

2005.03.10 SLIDE 6IS146 – SPRING 2005

Technosocial Situations

• Social setting as merging of technology, practice, place, and social relations– Goffman in the networked world– ….or ethnography of the technosocial

• Keitai use as technosocial practice– Merging of remote/mediated (virtual) relations and physically co-

present (real) relations– New practices and competencies for navigating different

situations and stages

• Keitai messaging technosocial situations– Mobile Text Chat– Augmented Flesh Meet– Ambient Virtual Co-Presence

Slide from Ito

2005.03.10 SLIDE 7IS146 – SPRING 2005

Emergent Technosocial Orders• Changing sense of co-presence

– Pulsating motion between background and foreground awareness and engagement

– Couple communication via mobile email particularly distinctive

• Move from serendipitous to intentional contact– Individual selectivity in communication and contact– Networked individualism (Wellman)– Full-time intimate community Yoshii et al)– Tele-cocooning (Habuchi)

• Portable practices– Layering of personal/intimate and place-based meanings– Urban space is personalized,no longer anonymous/alienated– We carry more of our identities and social connections around

with usSlide from Ito

2005.03.10 SLIDE 8IS146 – SPRING 2005

Cheskin Research: Methods and Goals• Methods

– Quantitative primary research with teens, age 13-18, and young adults, age 19-24

– Interviews with industry experts– Secondary research

• Motivations for wireless study– Youth influence larger consumer trends– Figure out new wireless product and service offerings

• Goals– Understand the phenomena being observed– Advise customers about what actions to take based

on that understanding– Sell these services and the value of their approach

2005.03.10 SLIDE 9IS146 – SPRING 2005

Cheskin Research: The Wireless Future

1. Social connectivity and entertainment will be the primary defining characteristics of wireless devices in the youth market, and likely the consumer market at large

2. Young people will build relationships via wireless devices

3. Multitasking capabilities will flourish within the youth market

5. Personal security and convenience will continue to be motivating factors for first time mobile phone consumers

6. Personalization of design, function, and interface will be a common expectation

7. Wireless entertainment and information applications will become favored "gap-fillers"

8. Strategic convergence will define the most successful wireless devices

9. Entertainment will drive the development of wireless cross-platform content

2005.03.10 SLIDE 10IS146 – SPRING 2005

Ito and Cheskin on Mobile Youth• How do their methods, motivations, and

goals compare?

• How do their findings compare?

• What factors affect the similarities and differences?

2005.03.10 SLIDE 11IS146 – SPRING 2005

Nick Reid on Ito and Okabe

• How do “social spaces” intersect? What I mean is, if there is a situation where a group of people are together, and through telepresence, how does this outside party enter into a group? How is their presence felt by people who are there or are not there, would it really seem like the person is in the next room?

2005.03.10 SLIDE 12IS146 – SPRING 2005

Nick Reid on Ito and Okabe

• A question that would be fun to discuss is what counts as “contact”? Is contact a hug or is contact a SMS? Does a communication not being “physical” demean the communication? “When a situation is predictable there is no information present.” Another question would be, does contact actually have to transmit information?

2005.03.10 SLIDE 13IS146 – SPRING 2005

Steven Lybeck on Ito and Okabe• Ito and Okabe show that new technologies

are spawning the creation of virtual social spaces that are quite analogous to physical ones. Could these virtual spaces supersede or even replace interaction in physical spaces? Why or why not?

2005.03.10 SLIDE 14IS146 – SPRING 2005

Steven Lybeck on Ito and Okabe• Are there any examples of virtual spaces

constructed without the use of new media technologies?

2005.03.10 SLIDE 15IS146 – SPRING 2005

Lecture Overview

• Review of Last Time– Social Uses of Mobile Phones

• Today– Reading Visual Representations

• Preview of Next Time– Midterm Exam

2005.03.10 SLIDE 16IS146 – SPRING 2005

Questions For Today

• What does it mean that images are constructed?

• How do we read images?

• Why are most of us not visually literate?

• How do we become visually literate?

• What do the previous questions mean for how we use, understand, and design cameras and photographs?

2005.03.10 SLIDE 17IS146 – SPRING 2005

Yukaghir Epistle

2005.03.10 SLIDE 18IS146 – SPRING 2005

Airplane Instructions

2005.03.10 SLIDE 19IS146 – SPRING 2005

2005.03.10 SLIDE 20IS146 – SPRING 2005

2005.03.10 SLIDE 21IS146 – SPRING 2005

Reading Images

• Visual communication is always coded– “It seems transparent only because we know the code already, at least

passively — but without knowing what it is we know, without having the means for talking about what it is we do when we read an image.”

• Our culture is moving from textual to visual– “Until now, language, especially written language, was the most highly

valued, the most frequently analyzed, the most prescriptively taught and the most meticulously policed code in our society.”

• Visual “literacy” is not taught and needs to be– “If schools are to equip students adequately for the new semiotic order,

if they are not to produce people unable to use the 'new writing' actively and effectively, then the old boundaries between 'writing' on the one hand, traditionally the form of literacy without which people cannot adequately function as citizens, and, on the other hand, the 'visual arts', a marginal subject for the specially gifted, and 'technical drawing', a technical subject with limited and specialized application, should be redrawn.”

2005.03.10 SLIDE 22IS146 – SPRING 2005

Semiotic Landscape

• Relationship between– The range of forms or modes of public

communication available in a society– The uses and valuations of these forms or

modes of public communication available in a society

2005.03.10 SLIDE 23IS146 – SPRING 2005

• What are the signifier, the signified, and the sign?• What are the similarities and differences between

linguistic signs and visual signs?

Semiotics Review

Signified

Signifier

“dog”“dog”

dog

2005.03.10 SLIDE 24IS146 – SPRING 2005

• Sign, Signified, Signifier – The linguistic sign is the unity of the signifier

(a sound-image) and the signified (a concept)

SaussureLinguistic Sign

Concept

Sound-Image

2005.03.10 SLIDE 25IS146 – SPRING 2005

The Linguistic Sign

“dog”

dog

2005.03.10 SLIDE 26IS146 – SPRING 2005

The Visual Sign

“dog”

2005.03.10 SLIDE 27IS146 – SPRING 2005

The Visual Sign

“dog”

2005.03.10 SLIDE 28IS146 – SPRING 2005

Arbitrariness of the Visual Sign

• Theories of visual denotation– Iconic (i.e., onomatopoetic)

• Images are an analogous reproduction of what they represent

– Arbitrary• Images construct an arbitrary relationship between

signifier and signified

– Motivated• The relationship between the signifier and signified

is motivated, but by what?– A “natural” analogy between image and the world?– By the conventions of visual language?

2005.03.10 SLIDE 29IS146 – SPRING 2005

Social Semiotic Theory: Metafunctions

• Ideational metafunction– To represent, in a referential or pseudo-referential

sense, aspects of the experiential world outside its particular system of signs

• Interpersonal metafunction– To project the social relations between the producer

of a sign or complex sign, and the receiver/reproducer of that sign

• The textual metafunction – To form texts, complexes of signs which cohere both

internally and with the context in and for which they were produced

2005.03.10 SLIDE 30IS146 – SPRING 2005

“Portrait”

2005.03.10 SLIDE 31IS146 – SPRING 2005

“Portrait”

2005.03.10 SLIDE 32IS146 – SPRING 2005

“Portrait”

2005.03.10 SLIDE 33IS146 – SPRING 2005

“Portrait”

2005.03.10 SLIDE 34IS146 – SPRING 2005

“Portrait”

2005.03.10 SLIDE 35IS146 – SPRING 2005

“Portrait”

2005.03.10 SLIDE 36IS146 – SPRING 2005

“Portrait”

2005.03.10 SLIDE 37IS146 – SPRING 2005

“Portrait”

2005.03.10 SLIDE 38IS146 – SPRING 2005

“Portrait”

2005.03.10 SLIDE 39IS146 – SPRING 2005

“Portrait”

2005.03.10 SLIDE 40IS146 – SPRING 2005

“Portrait”

2005.03.10 SLIDE 41IS146 – SPRING 2005

“Portrait”

2005.03.10 SLIDE 42IS146 – SPRING 2005

“Portrait”

2005.03.10 SLIDE 43IS146 – SPRING 2005

“Portrait”

2005.03.10 SLIDE 44IS146 – SPRING 2005

“Portrait”

2005.03.10 SLIDE 45IS146 – SPRING 2005

Natalie Torin on Visual Representations

• Kress and van Leeuwen wrote that "the dominant visual language is now controlled by the global cultural/technological empires of the mass media..... which exerts a 'normalizing' rather than explicitly 'normative' influence on visual communication across the world." Do you agree with this statement? How might this notion have changed in the last decade, considering that the article was written in 1995?

2005.03.10 SLIDE 46IS146 – SPRING 2005

Natalie Torin on Visual Representations

• Roland Barthes describes the special status of the photographic image as a "message without a code." Why do photographs have this special status yet pictorial images need a qualification?

2005.03.10 SLIDE 47IS146 – SPRING 2005

Natalie Torin on Visual Representations

• Any good semiotic system has to be able to project a particular social relation between the producer, the viewer, and the object represented. Can you think of an example of a very clear semiotic system? A very unclear semiotic system?

2005.03.10 SLIDE 48IS146 – SPRING 2005

Natalie Torin on Visual Representations

• Kress and van Leeuwen wrote that "some things can be said only visually, and others only verbally." Stemming from the notions of Ong's orality and literacy, is it possible to represent and understand something equally in either fashion? What gets lost in the process of converting from visual to verbal or vice versa?

2005.03.10 SLIDE 49IS146 – SPRING 2005

Lecture Overview

• Review of Last Time– Social Uses of Mobile Phones

• Today– Reading Visual Representations

• Preview of Next Time– Midterm Exam

2005.03.10 SLIDE 50IS146 – SPRING 2005

Review Session

• 6:30 pm – 8:00 pm

• 202 South Hall

• NO sections next week!

2005.03.10 SLIDE 51IS146 – SPRING 2005

Midterm Study Guide

• Midterm structure– Short answer questions– Ethnographic analysis question– LOGO programming analysis question

• Studying tips– Use midterm study guide– Study in groups– Be prepared to answer all questions on your

own