lessons learned from the netsense smartphone study aaron striegel 1 , shu liu 1 , lei meng 1 ,
DESCRIPTION
Lessons Learned from the NetSense Smartphone Study Aaron Striegel 1 , Shu Liu 1 , Lei Meng 1 , Christian Poellabauer 1 , David Hachen 2 , Omar Lizardo 2 1 Department of Computer Science and Engineering 2 Department of Sociology University of Notre Dame. Problem Space. - PowerPoint PPT PresentationTRANSCRIPT
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Lessons Learned from theNetSense Smartphone Study
Aaron Striegel1, Shu Liu1, Lei Meng1, Christian Poellabauer1, David Hachen2, Omar Lizardo2
1Department of Computer Science and Engineering2Department of SociologyUniversity of Notre Dame
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Problem Space
Wireless Data TsunamiSmall Cells
Indoor Coverage
IoT QoE / Analytics
Segmented IndustryStandards Bound
How do we get good data at a reasonable scale?
! 5x Grad Students + 1-3 Friends
NewDevices
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Why is it Hard?
PrivacyInvasive – Phone Agent
Great Data – Social, Location
ExpensiveRaw Cost - $60+ / month
Data Scale - Granularity
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Talk PremiseProvide a retrospective of the
NetSense study and futuredirections / study evolution
PhoneLab – Geoffrey ChallenSUNY Buffalo
LiveLabs – Archan MisraSingapore Management University
On-Going Efforts
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
NetSense Cell Phone StudyStudy how “always-on” networking (i.e. smartphones)
impacts how we make / keep friends
Provide 200 smart devices to incomingfreshmen at Notre Dame (Aug 2011)
Sociological
Proximity, Facebook,Phone Calls
Technical
Impact on Usage
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Funding “No bucks, no Buck Rodgers”
NSF SoCS grant - $750kThree years – 1 year prep + 2 years data200 users via “smart” devices
$75k fordevices
+ 200x Nexus S 4G
200 anytime mins
Unlimited data, text
=
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
ApprovalsHuman Subjects Institutional Review Board (IRB)
Consent Forms Deception vs. TransparencyWhat is being gathered? How often?
Data Security / AccessClear Policies – Overages, Breakages, Repairs
Collaborate / Educate Yourself Early
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
NetSense Agent – Year 1• Reflashed with Cyanogenmod• Gingerbread / ICS (2.4+)• “Full” root - monitoring• Proximity via Bluetooth• Mandated w/WiFi on
Agent application Local SQLite database Threads for “instruments” Periodically phones home
Communications SMS, Call, E-MailUsage Apps, TonnageEnvironment Battery, WiFi APs, B/T
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Study Mechanics
Distribute / manage 200 phonesReflash 200 phones w/CyanogenModDistribute 200 phones to the student
Week before classes
Cohort selectionNotification, responseND Dorm Assignment - July
Weekly meetings for 1x year
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Check-In Architecture
Next-generation Play Store basedXML Configurability, Rendezvous Server
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Active Devices Over Time(Mar 2012) Criterion
Must report w/ sufficient data (> 100 reports, on avg per day)
Threshold for evaluating “good” users who are fully compliant
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Active Devices (Mar 2012)
Coverage (indoor)plays a huge role
in battery life
Battery life istricky to balance
Provider upgradeswere
“challenging”
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Phone Repairs / EffortSD Card (Green) Peaked at end of 1st year Nexus S did not have removable SD card
Critical to havealways-on student / liaison to keep tabs
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Quiz Response RateMultiple choice quizzes Do you use WiFi? A. Yes B. No
Much worse response rates for long-style tie / behavioral quizzes
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
So what did you learn?
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Lessons LearnedPhone reimbursement / reward for participation
Purchased phone (subsidized) + free plan$100 (Galaxy S3), $200 (Galaxy S4)
Hardship cases?
Subsidy for compliance via pre-paid cards$40 / month for arm band, phone agent, laptop agent
Non-compliantusers?
Free phone + planUniversity-owned phones
Custom ROM?
Coercive?
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Lessons LearnedPoints of Contact are Key
Undergraduate liaisons S. Pentland @ MITPaid, < 10 hours per week
Configurable Sampling is EssentialProbabilistic, Triggered Sampling
Don’t Log It – GoneHow to Measure User QoE
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Lessons LearnedCohort EffectsHow big of a sample do you need?Is your population biased?Can you quantify the extent of bias?
Device BreadthIs a phone enough?How fine of data do you need?Per-packet RSSI? PHY layer? Tower info?
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Collaborating
Can I get access to your data?
Yes*Depending on what data you want
Can I run experiments on your phones?
Probably notSUNY PhoneLab
Singapore LiveLabs Urban Lifestyle Innovation Program
Cohort through May 2015 Cohort of +500 in August 2015*
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Potential for RelayingTo Appear at MobiCom 2013
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Overview• Problem Space• Overview of NetSense Study Goals• Study Process• Funding• IRB Approval• Architecture
• Response / Compliance• Lessons Learned / Looking Forward