smartphone applications: the next (and most important ...€¦ · gamification • concept:...

1

Copyright © 2012 The Nielsen Company. Confidential and proprietary.

Title of PresentationCopyright © 2012 The Nielsen Company. Confidential and proprietary.

Smartphone Applications:The Next (and Most Important?) Evolution in Data Collection

Presenter:

Michael W. LinkSVP/Chief Methodologist

Nielsen Institute for Measurement InnovationThe Nielsen Company

AAPOR 2012 ConferenceOrlando, FL

Michael W. Link, Jennie Lai, & Lorelle Vanno

2


Entering New Era of Data Collection

Manual Survey Collection Era (1930s-1970s)

Computer-Assisted Survey Collection Era

(mid 1970s-present)

Post-Survey Data Capture Era(2010 - ???)

3


New Worry: Respondent Expectations• Being involved in a scientific survey is a relatively rare occurrence

– Little, if any, expectations by respondents

• Respondent experience with new technologies is much different– Respondents are more savvy

• Developing expectations from these experiences– Ease of use/intuitive– Speed– Usefulness/utility or fun/entertainment or both– Ability to share experiences w others– Auto-features: ex. Location awareness

Need to recognize this societal shift and develop our new

measurement approaches accordingly

4


Critical Nature of User Engagement• “User engagement is the emotional,

cognitive, and behavioral connection that exists, at any point in time and possibly over time, between a user and a resource [i.e., technology]” (Attfield, et al, 2011)

• Encompasses non-utilitarian aspects –emphasis on understanding and designing for the subjective aspects of technology encounters

• It is essential that technology providers do not just design systems but that they design engaging experiences.

Source: Attfield et al, 2011http://www.dcs.gla.ac.uk/~mounia/Papers/engagement.pdf

5


Developing Measurement Systems for the New Landscape• How do we:

– Break old rules & develop new ones?– Leverage user engagement to improve participation?– Provide scientifically valid and reliable data to end-users/clients?

Form & Function

• Developing data collection tools which are ease to use, logical in flow, & visually appealing

Reciprocity

• Recognize data collection as two-way interaction and provide VALUE to respondent

Gamification

• Utilizing the psychological & sociological components of “game theory” and “game mechanics”

Social Sharing

• Facilitating greater respondent self-expression and potential content sharing with others

Four Potential Areas

6


Gamification• Concept: Identify sociological & psychological aspects of popular

game & entertainment apps, then apply them to measurement to improve user engagement and participation

– Does NOT mean literally turning measurement into a game

• Game Mechanics: various actions, behaviors, & mechanisms used to create a compelling user experience

• Game Dynamics: Motivations in response to experience that drives continued participation

Game MechanicsPointsLevelsChallengesVirtual goodsLeader boardsGifts & charity

Game DynamicsRewardsStatusAchievementSelf-expressionCompetitionAltruism

Interplay

7


Social Sharing• Ability to share information with others is ubiquitous (& expected) in

popular apps, measurement systems need to find ways to provide this & quantify measurement bias

• Violates long-held “rule” of measurement – measurement should be singular, not influenced by others

• Expression / respondent voice– To researcher – mechanisms to provide feedback, emotive measures

(beyond “Other: Specify”)– To others – share entries, views, behaviors, attitudes

– Other participants (family, community) or broader non-study groups

• Examples: Likes/dislikes, ratings, opportunities for commentaryPost comments to app-specific message feedPost information to SNSComment on posts of others

8


Example: Converting Paper Diary to Mobile App

9

Copyright © 2012 The Nielsen Company. Confidential and proprietary.AAPOR 2012

Friends & Family Test

Test Group Recruited RegisteredApp

CompletedFollow-Up

Full Start 100 90 67

Incremental 150 132 97

F&F Study Objective: Get application into as many hands as possible and develop some initial learnings to guide future research

• Sample: 250 Nielsen employees (non-probability, non-projectable)• Platform: iPhone only• Field Period: Jan 17 – Feb 27, 2012• $25 Gift Card Incentive

Research Program Goal: Develop a mobile alternative to Nielsen’s traditional paper TV Audience Measurement Diary

10


Respondent Engagement: Game Mechanics

• Goal Setting• InstructionBadges

• High Value Activities

• Redemption / Reputation

Points

• Top Performance• Social IncentiveLevels

11


BadgesHead Start Badge:You received the head start badge for completing your first full TV viewing log-in

Recall Badge:You received the recall badge for completing your first full past TV viewing log-in

Rebel Badge:You received the rebel badge for completing your first non-traditional (DVR /VCR, OnDemand or Online) TV viewing log-in.

Silver Cornerstone Badge:We could not get the data without you! You received the silver cornerstone badge for completion of 5 days of TV viewing.

Gold Cornerstone Badge:We could not get the data without you! You received the silver cornerstone badge for completion of 10 days of TV viewing.

12


Points & Statuses

Level

Total points needed to attain new level

0 TV Viewer <300

1 Grip 300

2 Best Boy 600

3 Gaffer 1,100

4 Fixer 1,600

5 Story Assistant 2,300

6 Editor 3,000

7 Assistant Director 3,900

8 Director 4,800

9 Show Runner 5,800

10Producer >7,000

Action for achieving points# points per action

Badge Received 300Survey Taken 200Promotion/Level 100Answered Push 25At least 1 Viewing Login that day 50

SSSSss

13


Respondent Engagement: Social Mechanics

• Status Updates• Influence

Social Sharing

• App Community• Facebook

CommunitySocial

Network

• TV Viewing• Status updates

“Likes” / Comments

14


Experimental Conditions Sample Size

Game Mechanics

Social Mechanics

Group #1Full App Week 1-6 100 Enabled Enabled

Group #2Incremental

Week 1-2

150

Disabled Disabled

Week 3-4 Enabled Disabled

Week 5-6 Enabled Enabled

Respondent Engagement Test GroupsTiming of Introduction of Game & Social Mechanics

15


Average Hours of TV Viewing Recorded per Week

Respondent Characteristics

Nielsen DiaryFeb ‘12

Group #1: Full App Group #2: Incremental App

(n)Weekly

Avg.Week 1

Avg.Week 6

Avg. (n)Weekly

Avg.Week 1

Avg.Week 6

Avg.

Total 26.9 (86) 6.9 10.4 4.7 (117) 7.9 10.7 6.8

18 – 34 yrs 17.5 (35) 6.8 9.5 5.1 (37) 8.2 10.4 8.1

35 – 49 yrs 25.9 (30) 7.2 11.0 4.0 (47) 6.8 10.1 5.8

50+ yrs 39.1 (19) 6.0 11.2 5.6 (31) 9.3 12.3 6.6

White 26.0 (66) 7.2 10.8 5.1 (88) 7.5 10.6 6.1

Black 36.4 (4) 5.9 6.3 3.3 (5) 16.9 20.8 14.1

Asian 20.5 (15) 6.0 10.0 3.8 (18) 7.7 9.1 8.6

Hispanic 25.8 (9) 5.9 7.8 3.8 (12) 7.2 9.1 6.7

Note: Includes all respondent who (1) downloaded the app and (2) recorded at least 1 hour of viewing during the 6 week field period.

16






(n)Weekly

Avg.Week 1

Avg.Week 6

Avg. (n)Weekly

Avg.Week 1

Avg.Week 6

Avg.

Total 26.9 (86) 6.9 10.4 4.7 (117) 7.9 10.7 6.8

18 – 34 yrs 17.5 (35) 6.8 9.5 5.1 (37) 8.2 10.4 8.1

35 – 49 yrs 25.9 (30) 7.2 11.0 4.0 (47) 6.8 10.1 5.8

50+ yrs 39.1 (19) 6.0 11.2 5.6 (31) 9.3 12.3 6.6

White 26.0 (66) 7.2 10.8 5.1 (88) 7.5 10.6 6.1

Black 36.4 (4) 5.9 6.3 3.3 (5) 16.9 20.8 14.1

Asian 20.5 (15) 6.0 10.0 3.8 (18) 7.7 9.1 8.6

Hispanic 25.8 (9) 5.9 7.8 3.8 (12) 7.2 9.1 6.7


17






(n)Weekly

Avg.Week 1

Avg.Week 6

Avg. (n)Weekly

Avg.Week 1

Avg.Week 6

Avg.

Total 26.9 (86) 6.9 10.4 4.7 (117) 7.9 10.7 6.8

18 – 34 yrs 17.5 (35) 6.8 9.5 5.1 (37) 8.2 10.4 8.1

35 – 49 yrs 25.9 (30) 7.2 11.0 4.0 (47) 6.8 10.1 5.8

50+ yrs 39.1 (19) 6.0 11.2 5.6 (31) 9.3 12.3 6.6

White 26.0 (66) 7.2 10.8 5.1 (88) 7.5 10.6 6.1

Black 36.4 (4) 5.9 6.3 3.3 (5) 16.9 20.8 14.1

Asian 20.5 (15) 6.0 10.0 3.8 (18) 7.7 9.1 8.6

Hispanic 25.8 (9) 5.9 7.8 3.8 (12) 7.2 9.1 6.7


18


Impact of Badges and Point & Levels Avg. Hours per Week per Person (Total Sample)

TV Viewing Hours per Week per Person

Badges, Points, LevelIncremental App

Source: Nielsen, Whatcha’ Watchin’ F&F Test Data, 2012

19


Impact of Badges and Point & Levels Avg. Number Days w 1+ Entries per Person (Total Sample)

Avg Number Days with 1+ Entry per Week per Person

Badges, Points, LevelIncremental App

Source: Nielsen, Whatcha’ Watchin’ F&F Test Data, 2012

20


Reasons for Use and Non-Use of Social Feed

0

10

20

30

40

50

60

70

80

90

100

Features UsedPush to FacebookCommentsLikesRead Entries

0

10

20

30

40

50

60

70

80

90

100

Why NOT Use SNSHard to TypeTime ConsumingNot Connected w CommunityNot Want to Share Viewing

N = 50 N = 70

21


Summary of Learnings To Date• Mobile platform has a ways to go before displacing current

methodology!

• Gamification is more important cross-time Little impact during first 2 weeksWhen introduced incrementally, impact is important but brief

• Social Mechanics had little impactUnclear if it is the way in which it was used or whether social

mechanics are simply not a useful engagement techniqueOrder effect?

• Key questions for all techniques: Tradeoff: enhanced participation vs potential for biasNot driving the attitudes or behaviors you’re trying to measure!

Study represents a first empirical “stake in the ground” – long road ahead to fully understand the best techniques for enhancing user engagement

22


Thank you!

Michael [email protected]

smartphone applications: the next (and most important ...€¦ · gamification • concept:...

Documents