towards individual and aggregate human behavior modeling...

Post on 23-Jul-2020

5 Views

Category:

Documents

0 Downloads

Preview:

Click to see full reader

TRANSCRIPT

Towards Individual and Aggregate Human Behavior Modeling from Data in the Real-World

Nuria Oliver, PhD

Scientific Director

User, Data and Media Intelligence

Telefonica Research

Outline

• Brief Introduction

• Mobile Phones as Human Behavioral Sensors

• Individual modeling: • Boredom Inference

• MobiScore

• Aggregate modeling: Big Data for Social Good

• Crime prediction

• Conclusions

• ~20 full time researchers

• Successful internship program with ~25

interns/year

• Successful stage program for undergraduate students

• Visiting professors and scholars

• Open innovation: Barca, Ferran Adria,

UNGP, MIT, Brno Univ, UCIrvine…• Hiring!!

3. Big Data Analytics 4. Human Computer

Interaction and

Mobile Computing

1. Machine Learning

Personalization and

User Modeling (RecSys)

2. Multimedia Data

Analysis (Voice)

User, Data and Media Intelligence

LinasBaltrunas

AlexandrosKaratzoglou

Jose SanPedro

Aleksandar Matic

Jordi LuqueXavier Anguera

Enrique Frias

Martin PielotSouneil Park

Joan Serra

Carlos Segura

Machine Learning Approachesto Model Individual and

Aggregate Human Trait and Behavior from a variety of

Data sources: voice, mobile sensors, serviceusage data, mobile network

data...

We work on…

Human Behavior @Telefonica

Summary

• Built capabilities over the years on:• Data Science, Machine Learning and Data Analytics:

• Mobile data, voice, text

• Personalization and Recommendations

• Human-Computer Interaction and Mobile Computing

• User Modeling

• Generated over 40 patents

• Achieved international recognition: • 6 best paper awards and 5 best paper award

nominations

• 3 prestigious Marie Curie Fellowships and 2 EU projects

• Rising Talent Award, IEEE and ACM Senior Member Award, 10-year technical award (ICMI)

• External influence: GSMA white paper for African operators re Ebola; ITU, UN, MWC presentations

Our Work in the Media

Our Work in the Media

Can mobilephones be used

to model, understand and help their usersand the world

at large?

Outline

• Brief Introduction

• Mobile Phones as Human Behavioral Sensors

• Individual modeling: • Boredom Inference

• MobiScore

• Aggregate modeling: Big Data for Social Good

• Crime prediction

• Conclusions

6.8 billion subscribers

96% of world’s population (ITU)

Mobile penetration of 120% to 89% of population (ITU)

Emerging and developed regions

More time spent on our phones than watching TV or with our with

our partner (US and UK)

6.8 billion subscribers

96% of world’s population (ITU)

Mobile penetration of 120% to 89% of population (ITU)

Emerging and developed regions

More time spent on our phones than watching TV or with our with

our partner (US and UK)

Detecting Boredom from

Mobile Phone Usage

Research

UbiComp ‘15, Osaka, Japan

Martin

Pielot

Telefonica

Research

Tilman

Dingler

University of

Stuttgart

Jose

San Pedro

Telefonica

Research

Nuria

Oliver

Telefonica

Research

times square night 2013. chensiyuan. Apr 16, 2013 via Wikipedia. CC BY-SA 4.0

War on Attention*

* http://www.forbes.com/sites/onmarketing/2012/10/19/the-attention-war/

SocialMediaCube. Yoel Ben-Avraham. Apr 8, 2013 via Flickr. CC BY-ND 2.0

The trade we make:

Our attention so Internet companies

can pay their bills

Our engagement is now defined by push-

driven notifications... We’re “hunting and pecking” through our app grid a lot less; the

apps that notify us are rewarded with our

engagement (and our dollars).

The Deluge of Push-Driven Notifications

‚Attention is a limited resource—a person has only so much of it ‘ [Matthew B. Crawford]

Attention Economy: treating human attention as a scarce commodity[Davenport and Beck, 2001]

times square night 2013. chensiyuan. Apr 16, 2013 via Wikipedia. CC BY-SA 4.0

Wild-West Land-Grab Phase

“Wild West Hotel, Calamity Av., Perry, 0. T., Sept. 93”. National Archives and Records Administration. Public Domain

Wild-West Land-Grab Phase

“Wild West Hotel, Calamity Av., Perry, 0. T., Sept. 93”. National Archives and Records Administration. Public Domain

If the trade

attention for free servicesis to be sustained

we need to better

protect mobile phone users

Could Boredom

be part of the solution?

Attention is not always scarce

Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY

2.0

Attention is not always scarce

Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY

2.0

Boredom displeasure caused by “lack

of stimulation”

[Fenichel, 1951]

“a bored person is not just someone who

does not have anything to do; it’s

someone who is actively looking for

stimulation”

[Eastwood, 2002]

Attention is not always scarce

Boredom displeasure caused by “lack of stimulation”

[Fenichel, 1951]

“a bored person is not just someone who does not have

anything to do; it’s someone who is actively looking for

stimulation”

[Eastwood, 2002]

Mobile phones are a commonly

used tool to kill time when bored

[Brown et al. 2014]

Attention is not always scarce

Mobile phones are a commonly used tool to

fill or kill time when bored [Brown et al. 2014]

Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY

2.0

Boredom displeasure caused by “lack of stimulation”

[Fenichel, 1951]

“a bored person is not just someone who does not have anything

to do; it’s someone who is actively looking for stimulation”

[Eastwood, 2002]

If phones knew when their

users are killing time

maybe they could suggest a

better use of that time

Is it possible to

detect

boredom from

mobile phone

usagepatterns?

Borapp – Sensor-Data Collection

Always

collected

Only

collected

if phone in

use

Sensor Description

Battery Status Battery level ranging from 0-100%

Notifications Time and type (app) of notification

Screen Events Screen turned on, off, and unlocked

Phone Events Time of incoming and outgoing calls

Proximity Screen covered or not

Ringer Mode Silent, Vibration, Normal

SMS Time of receiving, reading, and sending SMS

Sensor Description

Airplane Mode Whether phone in airplane mode

Ambient Noise Noise in dB as sensed by the microphone

Audio Jack Phone connected to headphones or speakers

Cell Tower The cell tower the phone is connected to

Data Activity Number of bytes up/downloaded

Foreground app Package name of the app in foreground

Light Ambient light level in SI lux units

Screen Orient Portrait or Landscape mode

Wifi Infos The WiFi network the phone is connected to

Experience Sampling

“Right now, I feel bored”

[5-point Likert scale]

Min. 6 times per day

Preferably triggered when

phone in use

Borapp – Experience Sampling

User Study: Data Collection

54 Participants

aged 21 – 46 (M = 30.6) years

11 female, 23male, 19 not disclosed

For two weeks in July 2014

Over 40M sensor log entries

4398 valid self-reports of boredom

0

500

1000

1500

0 1 2 3 4

Fre

qu

ency

Agreement to "Right now, I feel bored" 0 = disagree, 4 = agree

Boredom Ground Truth

0

500

1000

1500

0 1 2 3 4

Fre

qu

ency

Agreement to "Right now, I feel bored" 0 = disagree, 4 = agree

Absolute ground

truth

Bored: ratings 3, 4

446 (10.1%)

instances

Boredom Ground Truth

Absolute ground truth

Bored: ratings 3, 4

446 (10.1%) instances

Normalized ground

truth

Z-score per person

Bored: z > 0.25

1518 (34.5%) instances

0

400

800

1200

1600

2000

-2 -1 0 1 2

Fre

qu

en

cy

Normalized Subjective Boredom, (higher number = more bored than

usual)

Boredom Ground Truth

Category Example Feature Explanation

Context Semantic Location Home, work, other, unknown

Demographics Age, gender 38, female

Last Communication

Activity

Time last incoming

call

Time passed since somebody called the participants

Usage (intensity) Bytes received Number of bytes downloaded in the last 5 minutes

Usage (externally

triggered)

Number of

notifications

Number of notifications received in the last 5 minutes

Usage (idling) Number of apps Number of apps launched in the last 5 minutes

Usage (type) Most used app App used for the most time in the last 5 minutes.

35 Features, 7 Categories

RQ1: how well can phones detect

killing-time/boredom events from

these usage patterns?

RQ2: which usage patterns are

related to killing time with the

phone?

RQ3 is the model good enough to

be useful?

• Supervised machine learning classification: L2-

regularized logistic regression, linear SVMs and

Random Forests

• 5-fold cross validation

• Performance evaluation:

• Average Precision, Recall

• AUROC: area under the ROC curve information

about the ability of the model to rank users by their

probability to be bored

• Absolute and normalized boredom as ground

truth

Modeling Approach

74.6%

82.9%

0.0% 20.0% 40.0% 60.0% 80.0% 100.0%

normalized

absolute

Model Performance |

Random Forest (AUCROC)

74.6%

82.9%

0.0% 20.0% 40.0% 60.0% 80.0% 100.0%

normalized

absolute

Model Performance |

Random Forest (AUCROC)

Primary data set

74.6%

82.9%

0.0% 20.0% 40.0% 60.0% 80.0% 100.0%

normalized

absolute

Model Performance |

Random Forest (AUCROC)

Primary data set

62.4% precision for

50% recall of boredom events

Boredom can be detected

from phone-usage patterns

with an accuracy of ca.

75% to 83% AUCROC

Take Away #1

RQ1: how well can phones detect

killing-time boredom events from

these usage patterns?

RQ2: which usage patterns are

related to killing time with the

phone?

RQ3 is the model good enough to

be useful?

Recency of

communication activity

i.e., time since last

incoming or

outgoing

communication;

Feature Import Correlation The more bored, the ..

time_last_outgoing_call 0.0607 -0.143 less time passed

time_last_incoming_call 0.0580 0.088 more time passed

time_last_notif 0.0564 0.091 more time passed

time_last_SMS_received 0.0483 0.053 more time passed

time_last_SMS_sent 0.0405 -0.090 less time passed

time_last_SMS_read 0.0388 -0.013 more time passed

light 0.0537 -0.010 darker

hour_of_day 0.0411 0.038 later

proximity 0.0153 -0.186 less covered

gender (0=f, 1=m) 0.0128 0.099 more male (1)

age 0.0093 n.a. +20s/40s, -30s

num_notifs 0.0123 0.061 more notifs

time_last_notif_cntr_acc 0.0486 -0.015 less time passed

time_last_unlock 0.0400 -0.007 less time passed

apps_per_min 0.0199 0.024 more apps per minute

num_apps 0.0124 0.049 more apps

bytes_received 0.0546 -0.012 less bytes

bytes_transmitted 0.0500 0.039 more bytes

battery_level 0.0268 0.012 the higher

battery_drain 0.0249 -0.014 the lower

Recency of communication activity i.e., time since last incoming or

outgoing communication;

Phase of the dayi.e., hour of the day,

ambient light

Feature Import Correlation The more bored, the ..

time_last_outgoing_call 0.0607 -0.143 less time passed

time_last_incoming_call 0.0580 0.088 more time passed

time_last_notif 0.0564 0.091 more time passed

time_last_SMS_received 0.0483 0.053 more time passed

time_last_SMS_sent 0.0405 -0.090 less time passed

time_last_SMS_read 0.0388 -0.013 more time passed

light 0.0537 -0.010 darker

hour_of_day 0.0411 0.038 later

proximity 0.0153 -0.186 less covered

gender (0=f, 1=m) 0.0128 0.099 more male (1)

age 0.0093 n.a. +20s/40s, -30s

num_notifs 0.0123 0.061 more notifs

time_last_notif_cntr_acc 0.0486 -0.015 less time passed

time_last_unlock 0.0400 -0.007 less time passed

apps_per_min 0.0199 0.024 more apps per minute

num_apps 0.0124 0.049 more apps

bytes_received 0.0546 -0.012 less bytes

bytes_transmitted 0.0500 0.039 more bytes

battery_level 0.0268 0.012 the higher

battery_drain 0.0249 -0.014 the lower

Recency of communication activity i.e., time since last incoming or

outgoing communication;

Phase of the dayi.e., hour of the day, ambient light

Demographics,

i.e., gender and age;

Feature Import Correlation The more bored, the ..

time_last_outgoing_call 0.0607 -0.143 less time passed

time_last_incoming_call 0.0580 0.088 more time passed

time_last_notif 0.0564 0.091 more time passed

time_last_SMS_received 0.0483 0.053 more time passed

time_last_SMS_sent 0.0405 -0.090 less time passed

time_last_SMS_read 0.0388 -0.013 more time passed

light 0.0537 -0.010 darker

hour_of_day 0.0411 0.038 later

proximity 0.0153 -0.186 less covered

gender (0=f, 1=m) 0.0128 0.099 more male (1)

age 0.0093 n.a. +20s/40s, -30s

num_notifs 0.0123 0.061 more notifs

time_last_notif_cntr_acc 0.0486 -0.015 less time passed

time_last_unlock 0.0400 -0.007 less time passed

apps_per_min 0.0199 0.024 more apps per minute

num_apps 0.0124 0.049 more apps

bytes_received 0.0546 -0.012 less bytes

bytes_transmitted 0.0500 0.039 more bytes

battery_level 0.0268 0.012 the higher

battery_drain 0.0249 -0.014 the lower

Recency of communication activity i.e., time since last incoming or

outgoing communication;

Phase of the dayi.e., hour of the day, ambient light

Demographics, i.e., gender and age;

General usage

intensity i.e, phone out of

pocket, or time since

last phone use …;

Feature Import Correlation The more bored, the ..

time_last_outgoing_call 0.0607 -0.143 less time passed

time_last_incoming_call 0.0580 0.088 more time passed

time_last_notif 0.0564 0.091 more time passed

time_last_SMS_received 0.0483 0.053 more time passed

time_last_SMS_sent 0.0405 -0.090 less time passed

time_last_SMS_read 0.0388 -0.013 more time passed

light 0.0537 -0.010 darker

hour_of_day 0.0411 0.038 later

proximity 0.0153 -0.186 less covered

gender (0=f, 1=m) 0.0128 0.099 more male (1)

age 0.0093 n.a. +20s/40s, -30s

num_notifs 0.0123 0.061 more notifs

time_last_notif_cntr_acc 0.0486 -0.015 less time passed

time_last_unlock 0.0400 -0.007 less time passed

apps_per_min 0.0199 0.024 more apps per minute

num_apps 0.0124 0.049 more apps

bytes_received 0.0546 -0.012 less bytes

bytes_transmitted 0.0500 0.039 more bytes

battery_level 0.0268 0.012 the higher

battery_drain 0.0249 -0.014 the lower

Recency of communication activity i.e., time since last incoming or

outgoing communication;

Phase of the dayi.e., hour of the day, ambient light

Demographics, i.e., gender and age;

General usage intensity i.e, phone out of pocket, or time

since last phone use …;

Intensity of recent

usage i.e. # of unlocks, or

# of apps launched

in last 5 minutes, …

Feature Import Correlation The more bored, the ..

time_last_outgoing_call 0.0607 -0.143 less time passed

time_last_incoming_call 0.0580 0.088 more time passed

time_last_notif 0.0564 0.091 more time passed

time_last_SMS_received 0.0483 0.053 more time passed

time_last_SMS_sent 0.0405 -0.090 less time passed

time_last_SMS_read 0.0388 -0.013 more time passed

light 0.0537 -0.010 darker

hour_of_day 0.0411 0.038 later

proximity 0.0153 -0.186 less covered

gender (0=f, 1=m) 0.0128 0.099 more male (1)

age 0.0093 n.a. +20s/40s, -30s

num_notifs 0.0123 0.061 more notifs

time_last_notif_cntr_acc 0.0486 -0.015 less time passed

time_last_unlock 0.0400 -0.007 less time passed

apps_per_min 0.0199 0.024 more apps per minute

num_apps 0.0124 0.049 more apps

bytes_received 0.0546 -0.012 less bytes

bytes_transmitted 0.0500 0.039 more bytes

battery_level 0.0268 0.012 the higher

battery_drain 0.0249 -0.014 the lower

Apps

Co-occur with being

bored

Co-occur with

NOT bored

… and uncategorized apps

Boredom is related toRecency of communication

Phase of the day

Demographics

Intensity and type of phone

usage

Type of used apps

Take Away #2

RQ1: how well can phones

detect killing-time boredom

events from these usage

patterns?

RQ2: which usage patterns are

related to killing time with the

phone?

RQ3 is the model good enough

to be useful?

Borapp2

Model running on Mobile

Phone

Using primary data set

Constantly infers the

user’s boredom state on

the fly

Suggests Reading Buzzfeed Articles

User Study 2: Data Collection

16 Participants (different from 1st

study)

aged 18 – 51(M = 39) years

13 male, 2 female, rest did not disclose

For two weeks in Feb 2015

941 Buzzfeed recommendations

48% when predicted bored

Measure 1: Click-ratioFraction of times people

clicked on notification

(Mdn)

8% when not bored

20.5% when bored

(as inferred by the model)

Difference significant

z = -2.102, p = .018

Large effect

r = -.543

Measure 2: Engagement-ratio

Fraction of times people

spent more than 30 sec

reading (Mdn)

4% when not bored

15% when bored

(as inferred by the model)

Difference significant

z = -2.102, p = .018

Large effect

r = -.511

When inferred to be bored,

participants were …

More likely to clickMore likely to read

for > 30 seconds

The generic model was

powerful enough to

create significant, large

effects on click- and

engagement-ratios

Take Away #3

Impact in the Press…

Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY

2.0

Application Scenarios

Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY

2.0

Recommend

content to

alleviate

boredom

Shield user from non-important

interruptions during non-bored

times

Suggest useful but not

necessarily boredom-curing activities

Encourage embracing

boredom

Recommend content to

alleviate boredom

Shield user from

non-important

interruptions

during non-bored

times

Suggest useful but not necessarily boredom-curing

activities

Encourage embracing boredom

Recommend content to

alleviate boredom

Shield user from non-important

interruptions during non-bored

times

Suggest useful

but not

necessarily

boredom-curing

activities

Encourage embracing

boredom

Recommend content to

alleviate boredom

Shield user from non-important

interruptions during non-bored

times

Suggest useful

but not

necessarily

boredom-curing

activities

Encourage embracing

boredom

Being bored

is good for

you

Why don’t

you turn me

off?

Recommend content to

alleviate boredom

Shield user from non-important

interruptions during non-bored

times

Suggest useful but not

necessarily boredom-curing

activities

Encourage

embracing

boredom

Relevant Publications

http://doi.acm.org/10.1145/1864349.1864371

http://www.youtube.com/watch?v=_p7n_pn7xaE

TEDx Las Ramblas, Feb 2012

""When Attention is not Scarce: Detecting Boredom from Mobile Phone

Usage"

Pielot, M., Dingler, T., San Pedro, J. and Oliver, N.

Proc of ACM Int Conf on Ubiquitous Computing (Ubicomp 2015)

Best paper award!

"Boredom-Computer Interaction: Boredom Proneness and SmartPhone Use"

Matic, A., Pielot, M. and Oliver, N.

Proc of ACM Int Conf on Ubiquitous Computing (Ubicomp 2015)

Outline

• Brief Introduction

• Mobile Phones as Human Behavioral Sensors

• Individual modeling: • Boredom Inference

• MobiScore

• Aggregate modeling: Big Data for Social Good

• Crime prediction

• Conclusions

MobiScore: Credit Score Inference

With J. San Pedro, D. Proserpio & J. Gonzalez

Credit Scores, a number that represents an assesment or

the likelihood that a person will repay his or her debt, are

widely spread in the world enabling the growth of

consumer credit and transactional operations…

… They are calculated upon applicants’ creditand financial history…

… however, because of the data that is used to

compute credit scores, not everybody is

scorable…

Thin-file and no-hit:Not only a situation

for Emergent

Economies

2.5B

UNBANKED UNDERBANKED

8M

IMMIGRANTS

1.1M/year

NEW CLIENTS

No Positive Information

STUDENTS&YOUNG WORKERS

6.8 billion subscribers

96% of world’s population (ITU)

Mobile penetration of 120% to 89% of population (ITU)

Emerging and developed regions

More time spent on our phones than watching TV or with our with

our partner (US and UK)

6.8 billion subscribers

96% of world’s population (ITU)

Mobile penetration of 120% to 89% of population (ITU)

Emerging and developed regions

More time spent on our phones than watching TV or with our with

our partner (US and UK)

Would it be possible to build

an alternative score using

mobile behavioral data?

Credit

Score

Model

Probability of Default @30 and @90 days

Consumption

Social

Mobility

Mobile Behavioral

Data (CDR) MobiScore

Customer Relation

Management Data (CRM)

Demographic

Product

Socio-economic

Typical Mobile Behavioral Data• CDR

• SMS

Consumption Social Network Mobility

Call duration In/Out Degree Radius of gyration

N. Events Delta w.r.t time window Travelled distance

Lapse between events Unique Calls per day Rate of popular antennas

Reciprocated events Unique SMS per dayRegularity of popular

antennas

… … …

HR_ORG TLFN_A TLFN_B CD_GEO_A CD_GEO_B DT_ORG CD_SNTD CD_ERB CD_CCC QT_DUR

20:05:31 XXX YYY 3 11 20140519 2 1562 568 33

… … … … … … … … … …

HR_ORG TLFN_A TLFN_B CD_GEO_A CD_GEO_B DT_ORG CD_SNTD QT_TRFG

15:53:54 XXX ZZZ 3 25 20140506 2 1

… … … … … … … …

Customer Relation Management (CRM) Data

• Demographic information: age, gender

• Socio-economic• Derived from home address

• Payment variables: late payments

• Product features: device brand, device OS, device type, line type, line status, line quantity

• Time since activation

Credit default reports, i.e. pending

balance that is considered to be

uncollectible

More than 30 days in arrears

More than 90 days in arrears

Ground Truth

Credit

Score

Model

Probability of Default @30 and @90 days

Consumption

Social

Mobility

Mobile Behavioral

Data (CDR) MobiScore

Customer Relation

Management Data (CRM)

Demographic

Product

Socio-economic

Building Credit Scores from Data

• Mobile phone usage logs• 3-month period (Jan-March 2014)

• Fully anonymized

• Over 35 million call events and 11 million SMS events

• Customer Relation Management Data

• Credit default reports, i.e. pending balance that is considered to be uncollectible

• More than 30 days in arrears

• More than 90 days in arrears

Of over 60,000 individuals in a Latin American country

Mo

bile

Da

taFin

an

cia

l Info

Modeling Approach

• Supervised machine learning: L2-regularized logistic regression, linear SVMs and Gradient Boosted Trees

• 5-fold cross validation

• Performance evaluation:• Average Precision

• AUROC: area under the ROC curve information about the ability of the model to rank customers according to their probability of default

• Default @30 days and @90 days

2 weeks 1 month 3 months 2 weeks 1 month 3 months

GBT 63.0 64.5 67.5 68.5 69.4 71.6

@30 LR 62.2 64.0 66.4 67.6 68.8 70.7

SVM 62.1 63.9 66.7 67.9 68.8 70.6

GBT 63.1 64.4 67.5 70.2 70.8 72.5

@90 LR 62.4 64.5 67.4 68.7 70.5 72.1

SVM 63.1 64.1 67.2 69.7 70.3 72.1

Classification Performance (AUROC)

• GBTs outperform other classifiers thanks to their higher degree of complexity and flexibility

CRM + Voice CDRs CRM + Voice+SMS CDRs

Comparison with State of the Art

Implications

• MobiScore leverages passively collected mobile data to accurately infer default risk

• CDR + CRM features achieve significantly better performance than state-of-the-art models

• MobiScore opens the door to alternative credit score models that would enable millions of people to get access to credit

Relevant Publications

"MobiScore: Towards Universal Credit Scoring from

Mobile Data"

Proserpio, D., San Pedro, J. and N. Oliver

Proc. of Int. Conf on User Modeling (UMAP 2015)

"Prediction of Socioeconomic Levels using Cell Phone

Records", Victor Soto and Vanessa Frias-Martinez and

Jesus Virseda and Enrique Frias-Martinez, International

Conference on User Modeling, Adaptation and

Personalization, UMAP'11, Industrial Track, Girona, Spain,

2011

Outline

• Brief Introduction

• Mobile Phones as Human Behavioral Sensors

• Individual modeling:• Boredom Inference

• MobiScore

• Aggregate modeling: Big Data for Social Good

• Crime prediction

• Conclusions

6.8 billion subscribers

96% of world’s population (ITU)

Mobile penetration of 120% to 89% of population (ITU)

Emerging and developed regions

More time spent on our phones than watching TV or with our with

our partner (US and UK)

Cell Phones as Sensors of Human Activity

May 19, 2011, 7:06 pm The Sensors Are Coming!By NICK BILTON

Telecom / WirelessNEWSCellphones for ScienceScientists want to put sensors into everyone's hands

Digital footprints enable large-scale

analysis of human behavior

Sensors of aggregated human activities used to

monitor citizens’ interactions and mobility patterns (Song et

al. 2010; Dong et al. 2011)

understand individual spending behaviors (Singh et al.,

2013) and financial responsibility (San Pedro et al, 2015)

predict socio-economic indicators of territories (Eagle et al.,

2010; Soto et al., 2011; Smith-Clarke et al., 2014)

model spreading of malaria (Wesolowski et al., 2012) and

H1N1 (Frias-Martinez et al., 2011)

infer people’s traits (deOliveira et al. 2010, Staiano et al.,

2012; Chittaranjan et al. 2013; de Montjoye et al., 2013) and

states (Bogomolov et al., 2013)

Phones as Social Sensors

Big Data for Social Good

Crime Prediction

Analysis of impact of floods

http://www.wired.co.uk/news/archive/2013-10/17/nuria-oliver

Crime

Work with Bogomolov, A., Lepri, B., Staiano, J., Pianesi, F., Pentland, A.

Affects quality of life and economic

development both at the national and local level

Several studies explore relationships between

crime and socio-economic variables: education,

income, unemployment, ethnicity, …

Several studies have shown significant

concentrations of crime in small geographical

areas: crime hotspots

Crime

T1: Natural surveillance as key deterrent for

crime: people moving around are eyes on the

street (Jacobs, 1961)

high diversity among the population and

high number of visitors -> less crime

T2: Defensible space theory (Newman, 1972)

high mix of people -> more crime

Crime and Urban Environment

People-centric perspective vs Place-centric perspective

people-centric perspective used for

individual or collective criminal profiling

place-centric perspective used for

predicting crime hotspots

Crime Prediction

Data-driven and place-centric approach to

crime prediction

Multimodal approach: people dynamics

derived from mobile network data and

demographics

European metropolis: London

Prediction of crime hotspots and not criminals

profiling

Our Approach

Smartsteps Dataset:

for each of the Smartsteps cells a variety of demographic

and human dynamics variables were computed every

hour for 3 weeks (from December 9 to December 15, 2012

and from December 23, 2012 to January 5, 2013)

Criminal Cases Dataset: criminal cases for December 2012 and for January 2013

London Borough Profiles Dataset:

open dataset containing 68 metrics about the population

of a particular geographic area

Multimodal Approach: Data

• Footfall count: Shows the trend in footfall in a

specified area hourly, daily, weekly and

monthly. Provides a basic profile of the crowd.

• Catchment area: Shows which postal sectors

are your customers coming from by hour, day,

week and month. Shows the “battleground”

for two sites.

• Transport mode: Shows flows of crowds from

any two points, segmented by road, air, train,

etc.

SmartSteps

For each cell and for each hour the dataset contains:

an estimation of how many people are in the cell

the percentage of these people at home, at work or

just visiting the cell

the gender splits (male vs. female)

the age splits (0-20 years, 21-30 years, 31-40 years, …)

SmartSteps Data

Crime geolocation for 2 months (December 2012

– January 2013)

All reported crimes in UK specifying month and

year and not specific day/time

Median crime value (=5) used as threshold

Spatial granularity of borough profiles is at LSOA

levels: LSOA are small geographical areas

defined by UK Office for National Statistics (mean

population: 1500)

Crime Data

68 metrics about the population of a specific

geographical area: demographics, households,

migrant population, employment, earnings, life

expectancy, happiness levels, house prices, etc.

Spatial granularity of borough profiles is at LSOA

levels:

LSOA are small geographical areas defined by UK

Office for National Statistics (mean population: 1500)

London Borough Profiles Data

From Smartsteps data we extract

1st order features (mean, median, min.,

max., entropy, etc.)

2nd order features on sliding windows of

variable length (1 hour, 4 hours, 1 day,

etc.) to account for temporal patterns

Feature Extraction

Feature Selection

Mean decrease in Gini coefficient of

inequality

the feature with maximum mean decrease

in Gini coefficient is expected to have the

maximum influence in minimizing the out-of-

the-bag error

The feature selection process produced a

reduced subset of 68 features (from an initial

pool of about 6000 features)

Classification Approach

Binary classification task: high crime area vs low

crime area

10-fold cross-validation approach

Classifier: Random Forest (RF)

RF overcomes logistic regression, support vector

machines, neural networks, decision trees

Smartsteps-based classifier significantly outperforms baseline

majority and borough profiles-based classifiers

Experimental Results

ground-truth

Experimental Results

~70% accuracy in predicting crime hotspots

predictions

Features encoding daily dynamics have morepredictive power than features extracted on amonthly basis

Relevance of high number of residents to predictcrime areas

increased ratio of residents -> more crime (incontrast with Newman’s thesis)

Entropy-based features are useful for predicting thecrime hotspots

high diversity of functions (home vs work) and highdiversity of people (gender and age) act as eyeson street decreasing crime (in line with Jacobs’thesis)

Relevant Features

Only 6 out of 68 features in the joint model areLondon Borough features, namely

%working population claiming out of workbenefits

Largest migrant population

% overseas nationals entering the UK

% resident population born abroad

Relevant Features

Our method captures the dynamics of a

place rather than making extrapolations from

previous crime histories. We can use it in areas

where people are less inclined to report crimes

Our method provides new ways of describing

geographical areas: novel risk-inducing or risk-

reducing features of geographical areas

Implications

Relevant Publications“Moves on the street: Predicting Crime Hotspots using aggregated

anonymized data on people dynamics” - A. Bogomolov, B. Lepri, J. Staiano, Leouze, E., N. Oliver, F. Pianesi, A. Pentland Journal of Big Data (Big Data Journal 2015)

“Once Upon a Crime: Towards Crime Prediction from Demographics and Mobile Data” - A. Bogomolov, B. Lepri, J. Staiano, N. Oliver, F. Pianesi, A. Pentland 16th ACM International Conference on Multimodal Interaction (ICMI 2014)

"Flooding through the Lens of Mobile Phone Activity"Pastor-Escuredo, D., Torres Fernandez, Y., Bauer, J.M., Wadhwa, A., Castro-Correa, C., Romanoff, L., Lee, J.G., Rutherford, A., Frias-Martinez, V., Oliver, N., Frias-Martinez, E. and Luengo-Oroz, M. Proceedins of IEEE Global Humanitarian Technology Conference, GHTC 2014, Silicon Valley, CA, Oct 2014

Talk at WIRED 2013. London UK

http://www.wired.co.uk/news/archive/2013-10/17/nuria-oliver

Outline

• Brief Introduction

• Mobile Phones as Human Behavioral Sensors

• Individual modeling: • Boredom Inference

• MobiScore

• Aggregate modeling: Big Data for Social Good

• Crime prediction

• Conclusions

Conclusions

•Mobile Phones have huge potential to help their users

• Individual behavior modeling

• Persuasive computing, new services

• Mobile Phones have huge potential to help the world

• Mobile phones as sensors of aggregate human behavior

• Big Data for Social Good

A Few Challenges

• Representativeness of the data, generalization

• Combination of data from multiple sources

• Real-time analysis and prediction

• Lack of ground truth intervention to validate and attribute causality

• HCI challenges when designing IntelligentAssistants

• Regulatory and legal barriers

• Potential privacy risks and unintendedconsequences

Thanks!

nuria.oliver@telefonica.com

@nuriaoliver

We are hiring!

http://tidhiring.pdi.tid.es/

top related