cscw – evaluation p. dillenbourg & n. nova evaluation & exam

CSCW – Evaluation P. Dillenbourg & N. Nova

Evaluation & Exam


• 4 research positions• Project 1.

– D2: Visualization of group organization– D3: Statistical analysis – D4: Interactions analysis

• Project 2 – D5: Table design (11.01.2005)– D6: Video analysis ( PPT)

• Agenda– 11.01 Presentation D3 & D4 + Video Analysis– 18.01 The role of virtual & physical space in collaboration– 25.01 Knowledge Managment + Course evaluation– 01.02 Synthesis, Questions– ??.02 Exam

• 30 min: Team presentation of Project 2 (15 min) plus questions• 3 X 20 min individual questions on projects1 & 2• Language: English or French, up to you.

• Mark – 30%: (written) Deliverables D2,D3,D4– 30%: (oral) Presentation of project 2 Bring your PPT on laptop / USB key– 40%: (oral) Four questions on project 1 & 2 ( 50 question examples on the web site)– You can bring all documents and your laptop with you


http://dialogtable.com/


http://www.hc.ic.i.u-tokyo.ac.jp/project/Lumisight/


Hitachi

http://www.design.philips.comht

tp:/

/ww

w.w

ilkha

hn.f

r/


D6: Video Analysis: Questions1. Does the table support group participation?

– Does the leader occupy a more central position?– What’s the position of the ‘left over’ if any? – Are some chairs moves away from the table?– Do they exchange objects?– What’s the orientation of bodies and chairs?– Do they sit where you expected them to sit?

2. How laptop usage interact with other activities?– Do they move often their laptops for re-organizing their own space– Do they move their laptops or fold its screen to facilitate interaction?– How often do their turn their laptop to show its screen on the same-side partner or to

another-side partner ?– How much time is spent looking at the laptop versus looking at each other ?

3. How do they use the table space?– Is there some ‘dead’ zone versus an ‘interaction’ zone’?– What are the objects present on the table, which one are often used?– Do they share or exchange some objects?– What do they draw on the table? Where do they draw? Who draws?– Where do they put the other objects on the table (documents, mugs,..) ?– Is there useless space? Is there a ‘focus’ area?– Are there different phases in using the table space?


Video Analysis Methods

1. analytical(usability studies/ experimental research)

2. synthetic 3. critical events


http://www2.wcer.wisc.edu/Transana

Results

Segmentation

Coding

Counting

Transcript (annotated text file)

Units (utterances, turns, content units,…)

Categories (speech acts, grounding acts,…)

Transcription

Dialogue (audio file)

1. Detailed interaction analysis


Who draws where ?

123

4 5

6

• Colour = person

• Intensity = erase & redraw

Where are the laptops?

• A snapshot every 5 minutes

2. Synthetic usage maps


Who talk to who ?

13

4

• Social Network Analysis in a spatial context

Position & direction of the chairs?

• A snapshot every 5 minutes

2

2. Synthetic usage maps


3. Critical eventsIt’s a subjective method, used in ethnography, that aims to detect specific episode that help better to understand the dynamic of team processes than neutral statistics

• Salient events– Main reorganisation of space– Conflict about space (territories)– “Perspective” episode (turning documents or laptops)– Loss of participation– …

• Significant events– Emergence of convention on how to use space, implicit or explicit– Convergence towards a visual focus zone– …


D6: Video Analysis: Deliverables

1. 2-3 Maps of Table Usage

2. 2-3 “Critical events” (Interesting episodes)

3. Conclusions1. Pros and cons regarding to your table design

2. Suggestions for design a CSCW table (included groupware)

8 slides

cscw – evaluation p. dillenbourg & n. nova evaluation & exam

Documents

cscw evaluation

nova slide

nova evaluation exam

nova http

table space

nova video analysis

nova hitachi http

table documents