responsiveness in im: predictive models supporting inter-personal communication daniel avrahami,...
TRANSCRIPT
Responsiveness in IM: Predictive Models Supporting Inter-Personal Communication
Daniel Avrahami, Scott E. HudsonCarnegie Mellon University
www.cs.cmu.edu/~nx6
Q: if an instant message were to arrive right now, would the user respond to it? in how long?
collected field data 5200 hours 90,000 messages IM and desktop
events
models predicting responsiveness as high as 90.1%
why should we care?
why should we care?
IM is one of the most popular communication mediums no longer a medium just for kids (work / parents)
sending messages is “cheap” but the potential for interruptions is great
unsuccessful communication can have a negative effect on both sender and receiver can disrupt the receiver’s work can leave the sender waiting for information true not only for IM
how can such models help?
sender receiver
intercept alert mask enhance
awareness
message
sender
how can such models help?
message
receiver
intercept alert mask enhance
sender
how can such models help?
message
receiver
intercept alert mask enhance
sender
how can such models help?
message
receiver
intercept alert mask enhance
sender
how can such models help?
awareness
receiver intercept alert mask enhance
shhhh
sender
how can such models help?
awareness
receiver
intercept alert mask enhance (carefully)
not now
related work
instant messaging [Nardi’00 , Isaacs’02 , Voida’02]
interruptions and disruptions [Gillie’89 , Cutrell’01 , Hudson’02 , Dabbish’04]
models of presence and interruptibility [Horvitz’02 , Begole’02 , Hudson’03 , Begole’04, Horvitz’04 ,
Fogarty’05 , Iqbal’06]
coming up…
data collection participants responsiveness overview predictive models
how (features and classes) results
a closer look (new! not in the paper) future work
data collection
a plugin for Trillian Pro (written in C) non-intrusive collection of IM and desktop events
data collection (cont.)
privacy of data masking messages
for example, the message “This is my secret number: 1234 :-)” was recorded as “AAAA AA AA AAAAAA AAAAAA: DDDD :-)”.
alerting buddies hashing buddy-names
4 participants provided full content
participants
16 participants Researchers: 6 full-time employees at an
industrial research lab (mean age=40.33) Interns: 2 summer interns at the industrial
research lab (mean age=34.5) Students: 8 Masters students (mean age=24.5)
nearly 5200 hours recorded over 90,000 messages
responsiveness
0 50 100 150 200 250 300 350 400 450 500
Message Number
Day
Hour
10 min5 min2 min1 min30 sec
50%
responsiveness
0 50 100 150 200 250 300 350 400 450 500
Message Number
Day
Hour
10 min5 min2 min1 min30 sec
92%50%
defining “IM Sessions”
0 50 100 150 200 250 300 350 400 450 500
Message Number
Day
Hour
10 min5 min2 min1 min30 sec
session
92%
defining “Session Initiation Attempts”
0 50 100 150 200 250 300 350 400 450 500
Message Number
Day
Hour
10 min5 min2 min1 min30 sec
used two subsets: 5 minutes (similar to Isaacs’02) and 10 minutes
session
features
for every message: features describing IM state. including:
Day of week Hour Is the Message-Window open Buddy status (e.g., “Away”) Buddy status duration Time since msg to buddy Time since msg from another buddy Any msg from other in the last 5 minutes log(time since msg with any buddy) Is an SIA-5
features (cont.)
for every message: features describing desktop state (following Horvitz et al.
Fogarty et al. and others). including: Application in focus Application in focus duration Previous application in focus Previous application in focus duration Most used application in past m minutes Duration for most used application in past m minutes Number of application switches in past m minutes Amount of keyboard activity in past m minutes Amount of mouse activity in past m minutes Mouse movement distance in past m minutes
what are we predicting?
“Seconds until Response” computed, for every incoming message from a
buddy, by noting the time it took until a message was sent to the same buddy
examined five responsiveness thresholds 30 seconds, 1, 2, 5, and 10 minutes
modeling method
features selected using a wrapper-based selection technique
AdaBoosting on Decision-Tree models
10-fold cross-validation 10 trials: train on 90%, test on 10% next we report combined accuracy
results
79.883.8
87.089.4 90.1
0
10
20
30
40
50
60
70
80
90
100
30sec 1min 2min 5min 10min
Predict response within
% A
ccu
rate
results (full feature-set models)
all significantly better than the prior probability (p<.001)
results (user-centric models)
previous models used information about the buddy (e.g., time since messing that buddy)
can predict different responsiveness for different buddies but what if you wanted just one level of
responsiveness?
built models that did not use any buddy-related features
79.882.5
87.0 89.4 89.3
Use
r C
entr
ic
Use
r C
entr
ic
Use
r C
entr
ic
Use
r C
entr
ic
Use
r C
entr
ic
0
10
20
30
40
50
60
70
80
90
100
30sec 1min 2min 5min 10min
Predict response within
% A
ccu
rate
results (user-centric models)
all significantly better than the prior probability (p<.001)
a closer look
(new! not in the paper)
a closer look (new! not in the paper)
analysis of the continuous measure:
log(Time Until Response)
repeated measures ANOVA
Independent Variables: features subset
ParticipantID [Group] as random effect
DF DFDen F p
Group 2 1 0.01 0.992
HourMinutes 1 1257 1.32 0.251
log(ownStat_dur) 1 1564 1.62 0.203
log(timeSincOMsgBdy) 1 1212 7.63 0.006 *
log(timeSincOOther) 1 1416 1.50 0.221
log(buddyStat_dur) 1 1504 0.19 0.667
BaseRelationship 2 1447 1.03 0.357
MessageWindowsCount 1 1462 3.89 0.049 *
FocusedWindowType 18 971 1.47 0.093
FocusedWindowDur 1 1407 0.04 0.840
PrevFocusedWinFeatureDur 1 1567 9.45 0.002 *
MostFocusedWinTime(30) 1 985 0.03 0.865
MostFocusedWinTime(600) 1 957 0.49 0.485
WinSwitchesCountFeature(30) 1 1011 3.99 0.046 *
WinSwitchesCountFeature(600) 1 1089 0.97 0.326
MostFocusedWinType(60) 16 967 1.42 0.122
MostFocusedWinType(300) 20 1026 1.62 0.042 *
MouseEventCountFeature(30) 1 1046 2.98 0.085
MouseDistanceFeature(60) 1 1081 5.08 0.024 *
MouseDistanceFeature(600) 1 1567 1.60 0.206
KBCountFeature(30) 1 996 10.80 0.001 *
KBCountFeature(600) 1 1160 1.99 0.158
Estimate DF DFDen F p
Group 2 1 0.01 0.992
HourMinutes 1 1257 1.32 0.251
log(ownStat_dur) 1 1564 1.62 0.203
log(timeSincOMsgBdy) -0.06852 1 1212 7.63 0.006 *
log(timeSincOOther) 1 1416 1.50 0.221
log(buddyStat_dur) 1 1504 0.19 0.667
BaseRelationship 2 1447 1.03 0.357
MessageWindowsCount 0.08298 1 1462 3.89 0.049 *
FocusedWindowType 18 971 1.47 0.093
FocusedWindowDur 1 1407 0.04 0.840
PrevFocusedWinFeatureDur 0.00001 1 1567 9.45 0.002 *
MostFocusedWinTime(30) 1 985 0.03 0.865
MostFocusedWinTime(600) 1 957 0.49 0.485
WinSwitchesCountFeature(30) -0.16685 1 1011 3.99 0.046 *
WinSwitchesCountFeature(600) 1 1089 0.97 0.326
MostFocusedWinType(60) 16 967 1.42 0.122
MostFocusedWinType(300) Nom 20 1026 1.62 0.042 *
MouseEventCountFeature(30) 1 1046 2.98 0.085
MouseDistanceFeature(60) -0.00001 1 1081 5.08 0.024 *
MouseDistanceFeature(600) 1 1567 1.60 0.206
KBCountFeature(30) -0.00372 1 996 10.80 0.001 *
KBCountFeature(600) 1 1160 1.99 0.158
Estimate DF DFDen F p
Group 2 1 0.01 0.992
HourMinutes 1 1257 1.32 0.251
log(ownStat_dur) 1 1564 1.62 0.203
log(timeSincOMsgBdy) -0.06852 1 1212 7.63 0.006 *
log(timeSincOOther) 1 1416 1.50 0.221
log(buddyStat_dur) 1 1504 0.19 0.667
BaseRelationship 2 1447 1.03 0.357
MessageWindowsCount 0.08298 1 1462 3.89 0.049 *
FocusedWindowType 18 971 1.47 0.093
FocusedWindowDur 1 1407 0.04 0.840
PrevFocusedWinFeatureDur 0.00001 1 1567 9.45 0.002 *
MostFocusedWinTime(30) 1 985 0.03 0.865
MostFocusedWinTime(600) 1 957 0.49 0.485
WinSwitchesCountFeature(30) -0.16685 1 1011 3.99 0.046 *
WinSwitchesCountFeature(600) 1 1089 0.97 0.326
MostFocusedWinType(60) 16 967 1.42 0.122
MostFocusedWinType(300) Nom 20 1026 1.62 0.042 *
MouseEventCountFeature(30) 1 1046 2.98 0.085
MouseDistanceFeature(60) -0.00001 1 1081 5.08 0.024 *
MouseDistanceFeature(600) 1 1567 1.60 0.206
KBCountFeature(30) -0.00372 1 996 10.80 0.001 *
KBCountFeature(600) 1 1160 1.99 0.158
“those in the back can’t see, and those in the front can’t understand…”
Robert Kraut
a closer look (new! not in the paper)
work fragmentation longer time in previous app …. slower more switching (30sec) …. faster longer mouse movements (60sec) …. faster more keyboard activity (30 sec) …. faster more message windows …. slower
longer time since messaging with buddy… faster buddy ID had significant effect
implications for practice
(in the paper)
implications for practice
preserving plausible deniability
making predictions about the receiver, visible to the receiver
multiple concurrent levels of responsiveness
presented statistical models that accurately predict responsiveness to incoming IM based on naturally occurring behavior
we plan to examine using message-content to improve modeling
summary & future work
awareness
message interceptalertmaskenhance
we would like to thank
Mike T (Terry) James Fogarty Darren Gergle Laura Dabbish, and Jennifer Lai
this work was funded in part by NSF Grants IIS-0121560, IIS-0325351, and by DARPA Contract No. NBCHD030010
thank you
for more info visit: www.cs.cmu.edu/~nx6
or email: [email protected]
Feature Estimate F p
buddyName[Group,SN] 1.67 0.000 *
log(timeSincOMsgBdy) -0.06852 7.63 0.006 *
PrevFocusedWinFeatureDur 0.00001 9.45 0.002 *
MessageWindowsCount 0.08298 3.89 0.049 *
WinSwitchesCountFeature(30) -0.16685 3.99 0.046 *
MouseDistanceFeature(60) -0.00001 5.08 0.024 *
KBCountFeature(30) -0.00372 10.80 0.001 *
MostFocusedWinType(300) 1.62 0.042 *