ux research at napster: a product manager’s perspective

UX research at Napster: A Product Manager’s Perspective #UTwebinar @UserTesting

Suzanne ScharlockProduct Manager

Napster

2016 Napster / Rhapsody International. Company Confidential

ABOUT ME

SUZANNE SCHARLOCKCurrent: Product Manager at Napster. Building a new music streaming app.

Previous: Head of Product Marketing at Reveal Chat (formerly LikeBright).


Global Addressable Base of 1 Billion Carrier Customers & CE Devices

3,400,000+ Paying Subscribers

90% of Listeners use our Mobile Apps.

>$140MRoyalties Paid in 2015

80% Carrier

45% Paid Subscriber Growth

250Worldwide Staff

42,000,000 Songs and Growing

20% D2C

2014

2015

NAPSTER SNAPSHOT


HOW DO YOU KNOW YOU ARE BUILDING THE RIGHT PRODUCT?

4


UX STRATEGY

IDEATION SURVEYINGUSER TESTINGREVIEW AND REVISE RESULTSIMPLEMENTMEASURE REVIEW AND REVISE RESULTS


UX STRATEGY: IDEATION

PICK A KPI TO IMPROVEBe specific with the user behavior you want to improve. Ex: 15% of users will create a playlist.

BRAINSTORM IDEASEx: More users will create a playlist if the icon to open the playlist maker is on the homescreen

REVIEW PREVIOUS RESULTSLook at previous data to find patterns of behavior where users are more likely to hit the KPI.

Ex: Users who open the PL maker multiple times in a session are more likely to make a playlist.


UX STRATEGY: SURVEYING

CREATE HYPOTHESES Write down assumptions and hypotheses. Write survey questions to incorporate hypotheses.

Ex: Users make playlists to listen to musicUSE A SURVEYING TOOLRun a survey using Amazon Mechanical Turk or Survey Monkey. Use surveys to understand user desire and preference. Ex: When was the last time you made a playlist, what was it for?

SUMMARIZE AND DISTRIBUTE FINDINGSSummarize the survey findings in a report and distribute the report to relevant people throughout the company. Use the survey findings to shape the product requirements. Ex: Users often make playlists based on mood and activity, incorporate this into feature


UX STRATEGY: USER TESTING

CREATE PROTOTYPE / WRITE SCRIPTCreate a prototype and write a script for the test. Incorporate hypotheses in the testing script.

Hypothesis ex: Users can open the playlist maker button

IN-PERSON USER TESTINGConduct in-person user testing with at least 5 members of the target market.

REMOTE USER TESTINGConduct remote user testing with at least 5 participants on usertesting.com.


UX STRATEGY: REVIEW AND REVISE

WRITE AND DISTRIBUTE USER TEST SUMMARY Review all user test data and combine into a summary. Distribute summary throughout company/team.Ex: Discover that participants did not discover Playlist Maker icon

REVIEW RESULTSReview key learnings with product team members. Revise product requirements and feature design based on learnings.Ex: Review summary with product team, discuss new icon choices

REVISE PRODUCT AND RETESTCreate new feature iteration and re-test either in-person or remotely. Ex: Devise new Playlist Maker icon, create a new test

9


UX STRATEGY: IMPLEMENT

CREATE HI-FIDELITY VISUAL DESIGNS/REDLINESWork with visual designer to create hi-fi designs, redlines, icon fonts, assets etc.

Ex: Add new Playlist Maker icon to icon font, specify icon dimensions per device types etc.WRITE FEATURE REQUIREMENTS Create feature requirements including listing out KPIs for the feature and any needed metrics requirements.

Ex: Write feature to swap icon, determine needed metric events for icon REVIEW FEATURE REQUIREMENTS WITH DEV TEAMSReview with development leads first to discover any holes in feature requirements, revise feature requirements, then review with entire development teams.

Ex: Review icon change with dev teams

10


UX STRATEGY: MEASURE

MEASURE KPIS THROUGH METRICS TOOLReview KPI results on metrics tool such as Amplitude.

Ex: Measure open rate for Playlist Maker on Amplitude, determine if KPI was met

OBSERVE USER BEHAVIOR ON USERTESTING.COMUpload mobile build to usertesting.com, write a script/hypotheses and verify that the feature is usable in production.

Ex: Observe if participants open Playlist Maker via usertesting.com

11


UX STRATEGY: REVIEW AND REVISE

REVIEW RESULTSReview amplitude and usertesting.com results for the feature. Assess success or failure through pre-determined KPI thresholds. Revise product if necessary.

12


TEAM ORGANIZATION

PRODUCT TEAM3 members 1 UX designer , 1 visual designer, 1 product manager

MOBILE DEV TEAMSIncludes: iOS developers, Android developers, QA testers, Program Manager

API AND PLATFORM DEV TEAMS

13


TYPES OF RESEARCH

14

IN-PERSON USER RESEARCHVisiting students at a local college

REMOTE USER RESEARCHUsertesting.com

SURVEYING USERS/TARGET MARKETApptentive, Mechanical Turk


UX RESEARCH AND PLANNING

PRIORITIZE UX PROJECTS BASED ON KPISKPIs are organized into 5 Phases

1. Core Listening Use Cases 2. Premium Feature Use3. Monetization4. Onboarding Optimizations 5. Growth

15


UX RESEARCH AND PLANNING

ORGANIZE FEATURE WORK

16

Feature/Update Screen or Flow Bug, Feature, or Usability Update

Confirmed Work Rally Card Beta Priority Group

Beta Priority Description

Phase Platform Reasoning

Playlist Maker

Playlist Maker Feature Yes Link to rally

card 1.1 Listen to Playlists

1-Implement during beta iOS, Android

Users create playlists to listen to


PRODUCT/DEVELOPMENT KANBAN CYCLE

17

PO Backlog PO Definition Leads Review Meeting PO Refinement LeadInvestigation

Queue of feature ideas Queue of in-progress feature ideas

Review features that are product complete with dev leads

Problems with features discovered and features are handed back to product for review

Holes in features are fixed by product team and handed back to dev leads for review

Team Review Meeting PO Finalization Ready to take Platform Napster API

Features are reviewed with all client teams, QA teams, and platform/API teams

Problems with features found in team review meeting are fixed by product team

Features are handed to dev teams for implementation

Features for platform team Features for API team

Client Queue Client Dev Testing Final Review KPI Review

Queue of features for client team

Features currently in development

Features currently in testing with QA

Features are reviewed by dev teams

Features are implemented and reviewed in amplitude for performance

Completed Not Implemented

Feature is complete Feature is abandoned


TEAMS WORKING TOGETHER

DELIVERABLESCompleted product requirements including feature rules and needed metrics, Hi-Fi visual designs and redlines

18


TEAMS WORKING TOGETHER

19

DEVELOPMENT TEAMSDevelopment teams expect a rationale behind “why” they are building a feature

MONETIZATION TEAMWants to know if a premium feature is desirable and usable

EXECUTIVE TEAMWants to know that product team is doing due diligence and creating features that will help grow the company


EXAMPLE: DOUBLE TAP TO LOVE FIRST SHOT

20


EXAMPLE: DOUBLE TAP TO LOVE ROUND 2

21


EXAMPLE: THIRD TIME’S THE CHARM

22


USING USERTESTING.COM

USABILITY OF IMPLEMENTED FEATURESTesting general features that have been implemented in the builds

COMPETITOR ANALYSISTesting usability of similar features

23


USERTESTING.COM BEST PRACTICES

WRITE VERY SPECIFIC TASKSInclude warnings for users not progress to the next task until they completed the instruction for the current task

USE SCREENING QUESTIONSEx: Testers must know how to turn airplane mode on their phone to be eligible for the test

ONLY DEPLOY A SINGLE TEST AT A TIMEDo a test run of your initital test to find any parts of the script that could have been confusing for testers. Watch the test to the end and fix any hang-ups in the script before spending 5 credits for a complete test

24

THANKYOU

#UTwebinar

https://www.napster.com/

mailto:[email protected]

ux research at napster: a product manager’s perspective

Internet