- 1 / 26 - 방갈로르연구소(SISO)11

Smart Analytics in

Smartphones

Satnam Singh, PhD

Samsung Research India -Bangalore

Fifth Elephant Conference, BangaloreJuly 13, 2012

Disclaimer: Talk is based on my personal views and knowledge gathered from open sources

- 2 / 26 - 방갈로르연구소(SISO)22

• What is Smart Analytics?

• Trends in Smart Analytics

• Why to do Analytics in Device?

• Case Study: Sensory Data Analytics

Outline

- 3 / 26 - 방갈로르연구소(SISO)33

Smart Analytics

- Analytics keeping end-user in mind

- Enable use cases to bring new experience, ease and

benefits to end-user

Buying habits

Location and time

Activity

Entertainment

User Presence

SensorData

User Data

SocialData

SNS Data, RSS feeds

Images, Videos, Music,

Call logs, SMS data

Browser data

- 4 / 26 - 방갈로르연구소(SISO)44

Smart Analytics in Smartphones

Sensor Data

- Enhance User Experience- Recommendations- Personalization

Social data

User data…

Analytics (Text Mining, Machine Learning, Signal Processing)

SensorData

User Data

SocialData

3rd Party Applications, Native Applications

- 5 / 26 - 방갈로르연구소(SISO)55

User Data Analytics- Trends

Breadcrumbs

• A Simple Timeline of your Day

• Everything happening at your places

• Offers and Deals for your favorite places

Radii

• Connecting Personality to Places

• Match the place's personality with users

personality to give the best recommendations

• Deliver movie-like game experiences,

videos, images and wallpapers

• Bring users into the film's story and world

Paramount Pictures - Star Trek Into Darkness

Qualcomm’s Gimbal Platform Applications

- 6 / 26 - 방갈로르연구소(SISO)66

Sensory Data Analytics - Trends

Galaxy S4 Sensors Multiple sensors,

Environment sensing

Activity Recognition [Sensor Platforms, Alohar

mobile, ActiServ]

- 7 / 26 - 방갈로르연구소(SISO)77

Analytics in Server vs. Device

Device-based Analytics - Privacy concerns are taken care of..

• It works even if no network !!

• Need predictive models to run close to real-time and

automatically deploy them

• Power and battery consumption should be kept under

control

Server-based Analytics is needed if the application is too

compute intensive for a smart phone

• Latency and data transfer cost

• Data must be communicated securely

• Authentication before any data transfer

- 8 / 26 - 방갈로르연구소(SISO)88

Case Study: Sensory Data Analytics

Activity Recognition: Detect walking, driving, biking, climbing stairs, standing, etc.

Activity

Recognition

Running Biking

Climbing stairs Walking

Sitting

1. If phone call comes then

Send an automated SMS to

call later

3. Do not refresh

location Save

battery power

2. If phone call then

increase ring tone

- 9 / 26 - 방갈로르연구소(SISO)9

Data Visualization – Raw Data & Activity (Class Variable)

[Ref] Rattle R Data Mining Tool

Bar Plot

Example of Accelerometer data

- 10 / 26 - 방갈로르연구소(SISO)1010

Activity Recognition - Steps

Feature Extraction

Time Series Data 43 Features

Mean for each

acc. Axis (3)

Std. dev. for each

acc. Axis (3)

200 samples (10 sec)

Avg. Abs. diff. from

Mean for each

acc. Axis (3)

Avg. Resultant Acc. (1)

Histogram (30)

ClassifierCART: Decision Tree

Classify the

Activity

[Ref] Gary M. Weiss and Jeffrey W. Lockhart, Fordham University, Bronx, NY

[Ref] Jordan Frank, McGill University

[Ref] Commercial API Providers: Sensor Platoforms, Movea, Alohar

- 11 / 26 - 방갈로르연구소(SISO)11

[Ref] Rattle R Data Mining Tool

Decision Tree

-Accuracy for general model~75%, >95%

personalized model using 10 seconds

training for each activity

-Accelerometer sensor is low power

consuming sensor

- Use other sensors to figure out where is

smartphone Enhance accuracy by 5-6%

- 12 / 26 - 방갈로르연구소(SISO)12

Activity Recognition: Engg. Challenges

“Design Considerations for the WISDM Smart Phone-based Sensor Mining Architecture,”

SensorKDD ’11, Fordham University

• Supervised models- problems in collecting user data

• Data sampling rate for each activity: o High sampling rate than needed waste CPU cycles,

o While low sampling rate degrade the performance

• App should work even if device is in hibernation mode

• Control SQLite database overheads

• Power consumption and real-time computations

• Benchmarking and user testing is a key challenge

• Global user – support multiple languages for any text

mining application

- 13 / 26 - 방갈로르연구소(SISO)13

• Fusion of data science and domain knowledge can bring new experiences for end-users

• Getting data analytics-based feature in product needs intense team effort between various stakeholders

Summary

Thanks!!

- 14 / 26 - 방갈로르연구소(SISO)1414

Backup Slides

- 15 / 26 - 방갈로르연구소(SISO)15

[Ref] Rattle R Data Mining Tool

…

ΣRandom Forest

Tree1 Tree2

Treen

Random Forest: An Ensemble of Trees

- 16 / 26 - 방갈로르연구소(SISO)16

Another Approach: Activity Recognition

Feature Extraction

usingPCA

Classification usingSVM

9 PCsClassify

the

activity

“Activity and Gait Recognition with Time-Delay Embeddings” Jordan Frank, AAAI Conference on

Artificial Intelligence -2010

McGill University