the elusive nature of context: why we need it and were we might find it

1

V E R Z U SW E L C O M E T O

The Elusive Nature of Context

Gail C. MurphyUniversity of British Columbia

Tasktop Technologies @gail_murphy

A more restrictive license has been chosen given use of licensed images.

2

“the situation in which something happens”

M e r r i a m - W e b s t e r D i c t i o n a r y

Context

3

Context-aware Computing

where you are

who you are with

what is nearby

4

Context at work

open recent

5

L i m i t e d : w h e r e , w h o a n d w h a t a p p r o a c h t o c o n t ex t

E n h a n ce d : i n t e ra c t i o n t ra ce s ca n ex t e n d u s e o f co n t ex t t o w o r k

F u t u r e : i n t e n t + co n t e n t m a ke s a p p l i ca t i o n s w o r k f o r h u m a n s

The Elusive Nature of Context Take-Aways

6

I n t e r a c t i o n I n t e n t + C o n t e n t

S u m m a r y

The Plan: Context

T o d a y

7

Mobile

Desktop

8

Mobile

Desktop

9

W h e r e Y o u A r e W h o Y o u A r e W i t h W h a t I s N e a r b y

10


context on mobile is largely about the “where”

11


mobile applications that provide “what” are often based on “where”

12

W h e r e Y o u A r e W h o Y o u A r e W i t h W h a t I s N e a r b ywhen context is about “who”

it is typically application-based

13

Mobile

only device based context between applications

14

Mobile

small steps to increased context between applications within vendor e.g., integration with Messages app in iOS 10

15

Mobile

Desktop

16

Desktop

context is about…

location

recent activity — files open — directories visited

17

Desktop

manual configuration of work context via spaces

18

Desktop

prevalent metaphor is search

working on this talk — search for files — open application — search email for details — find images using search — and so on…

19

Desktop

is it better when we can speak requests? different modality, same metaphor

20

limited context

slightly richer context

21

L i m i t e d : W h e r e , W h o a n d W h a t a p p r o a c h t o c o n t ex t

The Elusive Nature of Context Summary of Today

humans responsible for bulk of context

humans working for applications

shouldn’t this be the other way around?

22

I n t e n t + C o n t e n t

S u m m a r y

The Plan: Context

To d a y I n t e r a c t i o n

23

search stresses semantic memory

24

Humans work in episodes

24


working on Elusive Context

talk

24



talk

working on some other task

24



talk


talk

working on some other task

25

P R O J E C T D E S C R I P T I O N

Actvities and Task Switches (Session 1)

Time [minutes]

Subj

ect

0 30 60 90 120 150 180

T3

T2

T1

S4

S3

S2

S1

R4

R3

R2

R1

●●

●●

●●●●

●●●●● ● ● ●●

● ●●●●●●

●●●●

●●●●●●

51 activity switches10 task switches, 3 distinct tasks

●●

●●

●

●●●●

●●●●

●

●●

●●●●●●●●

●●●●●

●

●●●

●●●●●●●●●●●●●●●

●

●

●

●●●

●●

●

●●●●

●●●●●●

●●●●●●●●●●●●●●

●

● ● ●

● ● ●●

●●●

●

●●

●●

●●●

●

●●●●

●● ●

●● ●●●●●●●●●


●

●●●●●●

●●●●●●●●●●●●●

●●●●●

●●●●●●●●

●●●●●●●

●●●●●● ●

●●

●●●●●

●

●●●●●●

●●

●

●●●●●●●

●●●●●●●●

●●●●●●●●●●●

●●●

●● ●●

●●

●●●● ●●

●●●

●

●●●

●●

●●●●●●●●●●●

●●●●●

●●●

●●● ●●●

●

●

● 230 activity switches79 task switches, 4 distinct tasks

●●●●●

●●●●●●●●●●●●●●●●●

●●●

●●●●●●● ●

●●●●

●

●●● ●●

●●●●●

● ● ●


● ●●●●●●

●

●

●

●●●●●

●●●● ●●●●●●●

●

●

●●●

●●●● ● ●

●●

●●● ●

●59 activity switches20 task switches, 5 distinct tasks

● ●●●●●●●●●●●●●

● ● ●●

●●●

●●●●●● ●

●

●●●


●●

●●●●●●●●●

●●●●●●●●●●

●●●

●●●●●●●

●●

●●●

●●●

●●●●

●●●●●●●●●

●●●

●

●

●●●●●

●●● ●

●

●

●

●●●●

●

●

●●

●●●●●

●●●

●●●● ●●●

●●●●●●●●●●


●●●●

●●●●●

●

●●

●●●●●

●●●●● ●●

●●●●● ●●●

●●

●●●● ●

●●●●

●108 activity switches16 task switches, 5 distinct tasks

●●●●●

●●●●●●

●

●●● ● ●

●● ● ●

● ●●●● ●

●66 activity switches25 task switches4 distinct tasks

●

●●

●●●●●

●●

●

●

●

●●●● ●● ●

●

●●

●●

●●●

● ●● ● ●●

●●●

●102 activity switches61 task switches6 distinct tasks

●

●

●●●●

●

●●●●●●●●●●

●

●●●

● ●●●● ●

●●●●●●●●

●● ●●

●●●●●●● ● ●

●●●● ●

●

●●●●

●●96 activity switches28 task switches, 4 distinct tasks

●

●

●

●

●

●

Dev:VCDev:DebugDev:CodeDev:ReviewDev:TestAppDev:OtherBrowsingRelBrowsingUnrelMeetInformalMeetPlannedEmailPlanningReadWriteDocOther

Software development episodes

[Meyer, Fritz, Murphy, Zimmermann, FSE 2014]

26


What if we leverage episodic memory?

27

Mylyn: leveraging episodic memory

interest based on frecency

[Kersten and Murphy, FSE 2006]

27



result is a task context[Kersten and Murphy, FSE 2006]

28



29



start with previous task context

30



start with previous task context

31


31


field study showed software developers more productive with Mylyn

downloaded 2M times per month

32

Task context is also useful for knowledge workers

33


Task-focused UI (with task contexts) for knowledge workers

[Kersten and Murphy, AI Magazine, 2015]

34

f i e l d s t u d y l o n g i t u d i n a l s t u d y

8 participants ~10 days use ~3 tasks per day tasks were revisited

75% of participants mixed documents and web pages

high decay (20%) indicates relevance weighing matters

K n ow l e d g e Wo r ke r s

3 participants 300-600 days of use 16-24 tasks per day ~3-4 task revisits per day

decay also shows relevance weighting matters

35

Interaction provides richer

context

36

E n h a n ce d : i n t e ra c t i o n t ra ce s ca n ex t e n d u s e o f co n t ex t t o w o r k

The Elusive Nature of Context Summary of Interaction

tasks + interaction leverage episodic

memory

apps are starting to work for humans

37

S u m m a r y

The Plan: Context

To d a y I n t e r a c t i o n I n t e n t + C o n t e n t

38

Mylyn

optimized for individuals can share task contexts

must share same resource structure explicit shared statement of intent

39

Mylyn

optimized for individuals can share task contexts

must share same resource structure explicit shared statement of intent

what if we relax these two constraints?

40

let’s focus back on software development

41

“It is better to fail in originality than

to succeed in imitation.”

H e r m a n M e l v i l l e

DevelopersBusiness Analysts Testers

42

E a c h r o l e … b e n e f i t s f r o m s p e c i f i c a p p l i c a t i o n s

h a s a p a r t i c u l a r p e r s p e c t i v e


43

R o l e s h a v e t o w o r k t o g e t h e r

P R O J E C T D E S C R I P T I O Ns h a r e p r o j e c t c o n t ex t

44

Business Analyst

Business Analyst

44

Business Analyst

Business Analyst

Developers

44

Business Analyst

Business Analyst

Tester

Developers

44

Business Analyst

Business Analyst

Tester

Developers

Our client has refined how data will be accessed

44

Business Analyst

Business Analyst

Tester

Developers


Communication through shared requirements application

45

Business Analyst

Business Analyst

Tester

Developers



45

Business Analyst

Business Analyst

Tester

Developers



Rich data synchronization



46

Business Analyst

Business Analyst

Tester

DevelopersRich data synchronization



46

Business Analyst

Business Analyst

Tester

Developers

46

Business Analyst

Business Analyst

Tester

Developers

Content (data) is transferred Intent (semantics) comes from application

46

Business Analyst

Business Analyst

Tester

Developers

Content (data) is transferredIntent (semantics) comes from application

47

intent is implicit in each application

content is moved between applications

each role has information in context

no more searching emails, finding documents

more time doing what humans are good at

enable increased productivity

48


The Elusive Nature of Context Summary of Intent + Content

48



Intent + Content

48



Intent + Content

how does context change if intent + content can be recognized, represented and shared?

49

The Plan: Context

To d a y I n t e r a c t i o n I n t e n t + C o n t e n t

S u m m a r y

50

Daniel AlmeidaJohn AnvikDeepak AzadElisa BaniassadAlex BradleyRoberto BittencourtWesley CoehloDavor CubranicBrian de AlwisNeil ErnstMichalis Famelis Sylvie FossThomas FritzJan HannemannLyndon HiewReid HolmesRahul JiresalMik KerstenAlbert LaiGene LeeArthur MarquesShawn MintoEmerson Murphy-Hill Owen OuMarc PalyartSarah RastkarMartin RobillardIzzet SaferNick SawadskyDavid Shepherd Jonathan Stiansen Albert ThompsonApple ViriyakattiyapornGiovanni VivianiRobert WalkerAnnie YingTrevor YoungSedigheh Zolatkaf

Thanks to Tasktoptians, NSERC, IBM,Microsoft, ABB and other funders for making much of the underlying work possible

51

“the situation in which something happens”

M e r r i a m - W e b s t e r D i c t i o n a r y

Context

52

Apple Knowledge Navigator (1987)

https://www.youtube.com/watch?v=QRH8eimU_20

A lack of context means

53

we (humans) are working for applications instead

of applications working for us

54

richer notions of context can

enable applications to work for humans

All is not lost

55

L i m i t e d : W h e r e , W h o a n d W h a t a p p r o a c h t o c o n t ex t

E n h a n ce d : I n t e ra c t i o n t ra ce s ca n ex t e n d u s e o f co n t ex t t o w o r k

F u t u r e : I n t e n t + co n t e n t m a ke s a p p l i ca t i o n s w o r k f o r h u m a n s

The Elusive Nature of Context Take-Aways

@ g a i l _ m u r p h y