CRE QUARTERLY MEETINGSeptember 4, 2014

Welcome!

OPENING COMMENTS

Ceril Shagrin, CRE Chair

STEERING COMMITTEE

Pat Liguori

4

2. Nielsen CRE Participation: Consensus of Steering committee that Nielsenparticipation needs to improve.

• Need regular attendee to provide continuity on and across projects, and toreacquaint Nielsen management with value of CRE;

• Infrequent participation lack of perceived value funding reduction.

How to Solve:• Facilitator reaches out to Lynda Clarizio to arrange meeting with CRE Chair• Conduct presentation to client-facing Nielsen employees showing what studies/ presentations have been done and how to use them to help clients.

CRE Steering Committee

Members: Brad Adgate, Michele Buslik, Nancy Gallagher, Tanya Giles, Janice Finkel-Greene, Jed Meyer, Ceril Shagrin, Ira Sussman, Robin ThomasChair: Pat Liguori

Conference call on 8/13/14

1. Hadassa Gerber: Membership re-application unanimously approved.

“At all times, the Chief Research Officer and one other executive officer of Nielsen designated by Nielsen shall be members of the Council.”

--- CRE By-Laws, Article 1, Section 1.1

5

CRE Steering Committee

4. Active Membership: Already defined in CRE By-Laws:“Each member of the Council must participate in at least one committee of the Council and participate in a majority of meetings of such committee.”

--- CRE By-Laws, Article 2, Section 2.1

5. New Members: Steering committee recommends candidate for CRE membershipserve on a committee, participating in 2 committee meetings or conference callsprior to the taking of a membership vote.

3. Bill Payment: Delays in payment to vendors by Nielsen; Facilitator to discusswith Nielsen.

Conference call on 8/13/14

INSIGHTS TO PRACTICE

Nancy Gallagher

CRE FINANCES

GUEST

Steve Hasker, Nielsen

SOCIAL MEDIA

Beth Rockwood

LOCAL MEASUREMENT

Billy McDowell

RESEARCH PROPOSAL:A PREDICTIVE MODEL OF LOCAL TV RATINGS USING SUPERVISED MACHINE LEARNING.

Submitted to the Council for Research ExcellenceSeptember 4, 2014

PROBLEM

> Over 150 local TV markets are currently measured by Nielsen using only a paper diary.

> CRE has demonstrated that relatively small sample sizes render audience ratings unstable and nonresponse bias further compromises the accuracy of diary estimates.

> Broadcasters and advertisers in small markets lack reliable and valid metrics with which to plan and conduct business.

OPPORTUNITY

> The Nielsen National People Meter has 20,000 HHs and is adding an additional 2300 households in diary-only markets.

> By applying techniques of machine learning, it may be possible to use these data to estimate local market ratings with accuracy significantly greater than current diary sample methods.

SUPERVISED MACHINE LEARNING

> An artificial intelligence technique in which the computer is presented with example inputs and their desired outputs in order to learn a general rule, an algorithm that maps inputs to outputs

> Inputs: TV Household characteristics, TV station characteristics and demographic viewing in 20,000 People Meter Households from outside a local market.

> Outputs: ratings estimates from the specific local market.

> Machine learning will develop a fitted model based on a “training set” and assess predictive accuracy with a “test set” of Households.

RESEARCH TEAM:

> Vasant Dhar: NYU Stern School of Business. Data Scientist

> Tim Dolson: LORE Media Research (Formerly VP, Statistical Methods, Nielsen). Data Consultant

> Sandy Retsky: Independent. Database Programmer

> Richard Zackon: CRE, NYU. Project Manager

PROPOSAL

> A test in ten Local People Meter markets selected by the Committee to predict local ratings by QH for four months of activity (Feb, May, July and Nov 2013) using simultaneous People Meter data from outside each market to predict ratings inside each market .

> Compare predictions with actual People Meter estimates from within the market to assess the accuracy of the predictions.

> Premise: if an acceptable level of accuracy can be established in the test markets, this same level could be expected in diary markets.

DATA

> Nielsen will provide tuning and viewing data by QH for Households and persons for all People meter HH’s for February/May/July/Nov 2013.

> Nielsen will also provide Universe Estimates by market and demographic, geographic and psychographic data (e.g. Claritas) for each sample household.

ANALYSIS

> The research team will prepare the data for analysis by assessing its quality, analyzing distributions and performing appropriate transformations.

> This will be followed by assessing various machine learning methods for the problem and applying the resultant algorithm on data from respondents with known features.

> The test metric will be Live Demographic ratings by QH.

> Accuracy goals- Validity: Rating within 5%- Reliability: Reduce Sampling Error by 50%

DELIVERABLE

> The CRE will be provided with a final report which will describe the analytic process including algorithms, validation statistics and recommendations for further R&D improvements to be taken up by CRE or Nielsen.

TIMETABLE

> September 4: CRE Approval

> September 20: Final specs

> October 20: Nielsen provides data

> November 14: Initial progress report to Committee

> December 15: Final report

COST:

> The cost of the project is $160,000, 50% payable upon contract and 50% payable upon delivery of final report.

BENEFITS

If the results are successful:> Local TV markets can consider an affordable new currency, more stable than one

based on small samples and more valid than one based on diaries or STB’s. > Nielsen will be encouraged to make further refinements to the model we develop.> The industry will be encouraged to innovate with careful use of advanced analytics.> Council and committee members will have a front row seat to learn from the project.

If the results are unsuccessful or are inconclusive: > The industry will have established a benchmark level of predictive accuracy for

machine learning techniques, setting the basis for further improvement. > Council and committee members will have a front row seat to learn from the project.

WHY SHOULD CRE COMMIT TO THIS PROJECT?

> The study directly addresses a critical methodological issue, consistent with CRE core mission.

> The limitations of unreliable metrics in small markets is a huge problem for Nielsen’s local clients.

> The risk to reward ratio for an R&D effort like this is quite favorable.

> Research using Machine Learning builds the CRE identity for both innovation and, if successful, excellence.

24

QUESTIONS?

DIGITAL RESEARCH

Bryon Schafer

DIGITAL RESEARCH COMMITTEEEthnographic Research Update

September 4th, 2014

STUDY SUMMARY

28

STUDY SUMMARY

LONGITUDINAL ETHNOGRAPHY

National sample of 100 households studied over the past year, with another year of research starting in November

Project duration: Nov. 2013 to Oct. 2015

Initial 3 hour visits to 100 households nationwide

Continual tracking of all households via a Participant Toolkit (PTK)

Targeted, in-person time in various environments, inside and outside of home, with a subset of 25 households

Researchers will be visiting 25-50 new households in 2015

29

LONGITUDINAL VIDEO VIEWING STUDY UPDATE

PORTLAND

PHOENIX

CHICAGO

MEMPHIS

NEW YORKPHILADELPHI

A

MIAMI

KANSASCITY

30

2014 STUDY TIMELINE

TasksStudy Kicked off & Fully Recruited

Phase 1 Initial Ethnographic Visits

Household-Specific Assignments via Toolkit

Additional Ethno. Visits (Subset)

Phase 2Strategy for Year 2

Potentially Recruit Additional HHs

100 homes visited at the end of March

Jan Feb Mar Apr May

Jun Jul Aug Sep Oct Nov Dec

100 HHs

ALL 100 HHs

25 (20 Completed as of 8/1*)

Subset Of 25 Chosen

25-50 NEW HHs

Today DirectionFor Year 2

PRESENTATIONS

32

NIELSEN CLIENT EVENT Held on August 6th in New York

CRE Digital Committee members and GfK presented

Attendees:

>Approximately 100 in person

>82 online

Upcoming presentations:

>The Market Research Event (TMRE)

>Media Insights and Engagement Conference

33

QUESTIONS FROM THE AUDIENCEAudience questions in response to the presentation can be summarized as follows:

> What tradeoff’s occur as the result of the increased availability of On Demand and time-shifted content?

> How do device experiences shape viewing behaviors?

> What behaviors are most difficult to track with current measures?

> When a single user device is used for “Casting” content how can that be captured?

> How can the content of greatest interest be determined?

> Are there behaviors that reveal this prioritization?

PLANNING FOR YEAR TWOPROPOSAL FOR CRE

35

PLANNING FOR YEAR TWO>The first year of research focused on establishing a deep understanding on the ways in which American families consume video media.

>A focus was made to look at key questions related to the understanding of viewing such as Credential Sharing, Mobile Device Usage, Sharing and Learning, and Device Selection.

>The second year of research will focus on deepening the observations from year one, with an additional focus on the future of viewing and measurement.

>Viewing behaviors will be analyzed in terms of potential issues for measurement, tradeoffs with other onscreen media (such as browsing and social media) and in respect to what future behaviors we expect to see emerge from the expanding world of devices and OTT services.

36

A PROPOSAL>This proposal builds upon the learnings from the Media Technology Acceleration Study in order to deepen the qualitative understanding of changes in viewing behavior brought about by devices that provide expanded viewing options.

>This research will inform the Longitudinal study by investigating behavior and measurement challenges that occur as a result of the proliferation of Over the Top (OTT) devices and services.

>These investigations will also inform the study’s methodology by helping to inform observations, conducted both in person and remotely, as the study progresses.

37

EXTENDING THE ACCELERATION STUDY>Project Concept & Goals. The participants from the Acceleration Study will be revisited in order to observe and compare differences in behaviors and usage. This will allow us to observe the choices and changes in behaviors over time.

>Informing Measurement. By design, this research should inform important measurement considerations for the CRE and for Nielsen. For example: Factors that surround the frequency of device and service churn, the impact on viewing locations and measuring outside of the home, and opportunities to understand future measurement challenges

>Per Nielsen, this study paints a great picture of the ecosystem of American homes and how they have become so nuanced in terms of devices and their usage. It emphasizes the need for Nielsen to consider the complexity of the viewing environments today and helps Nielsen to educate clients. This study has inspired a team within Nielsen to begin working on a future study to help quantify the behaviors this study has uncovered.

38

METHODOLOGY>Audit. An audit will be performed for all devices purchased and services subscribed to, altered or cancelled since the end of the previous study.

>Ethnographic Observation. A 90 minute home observation and interview with household members will address the research objectives

>Assignments. At the conclusion of the ethnographic visit, each household will be given assignments to complete over the four week period, which can include diary activity and video journaling, narrated slideshows, and observation activities enlisting a household member as an extension of the research team

>Research Technology. Participant Toolkit iPods will continue to be leveraged in the facilitation of information and data collection.

39

TIMING>We propose that Acceleration Study extension visits begin during the week of October 6, providing approximately five months of device usage, also accounting for the time that might be required for families to adopt to the schedule of the new school year and the new fall TV viewing season:

Kick-off, Screener, Recruit, Scheduling 3 Weeks

In-home Visits 2 Weeks

Assignments Due 4 Weeks

Final Report December 5

>The entire project will occur over the span of six weeks, with four meetings proposed to guide the project and provide feedback

40

BUDGET>We request up to $125,000 for the execution and completion of this study in 2014:

GfK Estimate (+/- 10%)

>Preparation $15,168

>Ethnography Visits$62,595

>Reporting $22,470

>Recruiting $20,000

>T&E $2,880

>Total $123,113

THANK YOU

ROI

David Poltrack

GUEST

Dave Morgan, Simulmedia

BIG DATA

Stacey Schulman

MEDIA CONSUMPTION & ENGAGEMENT

Joe Abruzzo

MEDIA CONSUMPTION AND ENGAGEMENT COMMITTEE

Agenda for Alignment of Cross-Platform Media Exposure Measurement

47

COMMITTEE MEMBERS> Joseph Abruzzo (Chair)> Redjeb Shah (Vice Chair)> Tim Brooks> Joanne Burns> Michele Buslik> Laura Carpentier> Laura Cowan> Natalie Davis> Bill Duggan> Janice Finkel-Greene> Betsy Frank> Nancy Gallagher> Janet Gallent

> Hadassa Gerber> Tanya Giles> Greg Iocco> Diahanna Post> Tim Reilly> Ceril Shagrin> Ira Sussman> Susie Thomas> Bruce Tyroler> Emily Vanides> Jack Wakshlag> Candace Whye> Richard Zackon

AGENDA FOR ALIGNMENT OF CROSS-PLATFORM MEDIA MEASUREMENT

> Matrix comparison of audience measurement methodologies across 59 currently available studies

> Review of current media measurement practices entitled “Issues and Challenges With Cross-Platform Media Measurement Alignment: Research Review and Analysis” covering:

− Alignment of Metrics and Units of Analysis− Alignment of Methodologies− Alignment of Reporting− Dimensions of the Media Consumption Experience− Overview of Cross-Platform Measurement Initiatives

> Interviews with industry executives to understand the issues considered most important to audience measurement across media platforms.

49

INTERVIEWS WITH INDUSTRY EXECUTIVESQUESTION AREAS

> The objective:

− to identify the issues which experts identify as most important for the development of a set of widely accepted industry cross-media metrics for measuring and valuing media and advertising across media platforms.

> The survey:

− explores issues media companies are facing with regard to audience measurement across platforms

− attempts to assess current cross platform solutions and approaches, and the kind of information that is needed now and in the future for effective selling and buying of cross platform media

INTERVIEWS WITH 50 MEDIA INDUSTRY EXECUTIVES

> BROADCASTERS/NETWORKS (9)

> AD AGENCIES/MEDIA AGENCIES (9)

> ADVERTISERS (10)

> Research (6)

> Tech (4)

> MVPDS (2)

> RADIO/PUBLISHING (3)

> ORGANIZATIONS/ASSOCIATIONS (4)

> CONSULTANTS/OTHER (3)

52

INTERVIEW CANDIDATES

COMPANY NAMEContacted/Scheduled/

CompletedBROADCASTERS/NETWORKS (9)ABC Affiliate Bill Fine ContactedABC Networks Geri Wang/Adam Gerber ContactedDiscovery Communications Michael Greco/Scott Felenstein ScheduledESPN Glenn Enoch ContactedFox Audrey Steele/Toby Byrne ContactedNBC Linda Yaccarino/Barry Fischer ContactedTurner Broadcasting Donna Speciale/Dan Aversano ScheduledUnivision Kevin Conroy/Mehul Nagrani CompletedViacom Colleen Fahey-Rush CompletedWarner Bros. Syndication John O'Hara/Michael Orgera Completed

AD AGENCIES/MEDIA AGENCIES (9)Annalect Adam Gitlin ContactedAssemblly Catherine Warburton Contactedd expósito & Partners Gloria Constanza CompletedDroga5 Jonny Bauer ContactedGroupM Worldwide Rob Norman/Lyle Schwartz ContactedHavas Media/NY David Handelman CompletedIPG Media Lab Melvin Wilson CompletedStarcom MediaVest Group Kate Sirkin ScheduledUniversal McCann Huw Griffiths Completed

53

INTERVIEW CANDIDATES

COMPANY NAMEContacted/Scheduled/

CompletedADVERTISERS (10)American Express Company Joseph Bihlmier ContactedColgate-Palmolive Company Nadine McHugh ContactedConAgra Foods Inc. Fernando Arriola ContactedHome Depot Rex Conklin ContactedJP Morgan Chase Colleen Milway ContactedKellogg Company Aaron Fetters CompletedKimberly Clark Jeff Holecko ScheduledMasterCard Worldwide Benjamin Jankowski ContactedMillerCoors LLC Stevie Benjamin ScheduledThe Wendy's Company Tim Sullivan Scheduled

Research (6)comScore Josh Chasin CompletedKantar Jeff Boehme CompletedNielsen Michael Link CompletedRentrak Bruce Goerlich CompletedSimulmedia Dave Morgan ScheduledSymphony Advanced Media Charles Buchwalter Contacted

54

INTERVIEW CANDIDATES

COMPANY NAMEContacted/Scheduled/

CompletedTECH (4) AOL Jim Norton ContactedBrightroll Tim Avila ContactedCollective Joe Apprendi ContactedGoogle Sanas Ahari Contacted

MVPDS (2)Comcast Matt Strauss ContactedTime Warner Cable Marshall Jacobowitz Contacted

RADIO/PUBLISHING (3)Clear Channel Radha Subramanyam/Tim Spengler CompletedCondeNaste Scott McDonald CompletedEmmis Paul Brenner Contacted

ORGANIZATIONS/ASSOCIATIONS (4)AAAA Bill Tucker ScheduledANA Bob Liodice ContactedARF Gayle Fuguitt ContactedCIMM Jane Clarke CompletedIAB Sherrill Mane CompletedMedia Ratings Council George Ivie Contacted

55

INTERVIEW CANDIDATES

COMPANY NAMEContacted/Scheduled/

CompletedCONSULTANTS/OTHER (3)Sequent Partners Jim Spaeth CompletedIndependent Jack Wakshlag CompletedIndependent Jim Figura CompletedTRA (Formerly) Bill Harvey Contacted

TENTATIVE TIMELINE

Event Week ofDistribution of final draft of “Issues and Challenges With Cross-Platform Media Measurement Alignment” and matrix comparison

Today

Completion of final interviews 10/6/14Completion of summary of findings “white paper” 10/20/14Initial findings shared with the CRE 11/3/14Initial findings shared with the Interviewees 11/3/14Initial findings shared with representatives from ANA, IAB, ARF,MRC 11/10/14Initial findings shared at ARF Cross-Platform Conference (San Francisco November 13)

11/10/14

Draft of Research Agenda for Alignment of Cross-Platform Media Measurement (internal CRE)

12/8/14

Presentation of Research Agenda - Media Insights and Engagement Conference, San Diego, February 3-5, 2015

2/2/15

SAMPLE QUALITY

Ceril Shagrin

WHAT NIELSEN MEASURES

COMMUNICATIONS COMMITTEE

Emily Vanides

COMMUNICATIONS ACTIVITY UPDATE

VP

Director

Other

SeniorCEO

Sources: LinkedIn stats, based on self-reporting; and internal CRE data. All data as of 9/2/14

635 members

Newsletter 708

News Releases 585

Subscribers

Webinars 587

Weekly Clips New

ETHNOGRAPHIES NEWS COVERAGE

OTHER COVERAGE OF ETHNOGRAPHIES

RECENT CRE-FOCUSED GUEST COLUMNS FOR TRADE OUTLETS

FORTHCOMING EVENTS & COMMUNICATIONS

> Committee-related activities:

− Big Data: Rollout of Big Data primer, October− Announcing completed cross platform whitepaper− ROI update

> Other:

− September Newsletter− Additional Guest columns− Promoting Richard as spokesperson for CRE

MARK BRAFF, PR CONSULTANT

> Develop strategy for story pitches/press release distribution> Write press releases> Arrange one-on-one reporter meetings for Richard Zackon, representing CRE> Pitch stories> Edit newsletter copy

WEB SITE RE-DESIGN

> $15,000 was spent to reprogram and clean up existing code and move to new server with thoughts of redesigning in the Fall.

> Redesign to include a “New” look, making site more “User-Friendly”, increasing “Social Sharing” and including “Search” capabilities.

− Cost $56,000-$85,000 which includes site architecture, design including stock photography, HTML/CSS programming, Mobile/Tablet programming, Backend programming and integration, management, testing and Q/A

− New site to go live around by end of 2014

EDUCATION COMMITTEE

Jed Meyer

ACTIVITY UPDATE

> Recruiting new members – UM, Greater Media, RAB> Update on “Growing the Media Research Profession” plan – “to-do” list for

interested parties distributed by Gary Corbit> Internship update

− Hosted meeting on June 16th in NYC for MAIP 2014 interns “Past/Present/Future of Research”

− Presented Research update to AAF AdCamp Consumer Insights Workshop on 7/28

> Time Warner and BEA put on the 3rd Annual College Professor Thought Leadership Seminar on July 21/22 at the TW Media Lab in Time Warner Center. Heather Birks reports that 30 professors attended and Robin Garfield & Howard Shimmel presented Turner/CNN's College Outreach Program.

MULTICULTURAL ADVERTISING INTERN PROGRAM (MAIP) SEMINAR

COVERAGE OF MAIP EVENT

NEXT STEPS

> MAIP 2015: Asking Agency Stakeholders to budget for MAIP research interns for Summer 2015

> Recruitment of additional core contributors to continue advancement of the mission

− Especially in Media / Publisher sectors

> Develop ideas for a Fall event for “Intermediate / Advanced tracks” that incorporates feedback from June event and is more engaging & interactive

> Establish more connections with University professors

NEW BUSINESS: CRE AND NIELSEN R&D

ADJOURN