Introduction to Mobile AnalyticsAndrew OwensSeptember, 2013

Agenda

● Rules of Thumb

● The Mobile Ecosystem

● Third Party Solutions

● The Analytics Lifecycle

● Analytics in the Mobile Development

Process

● Standard Reports & Dashboards

● Statistical Techniques & Tools

Out of Scope Items

● Mobile Web Analytics

● Covering all mobile analytics tools

● The mathematics behind A/B Testing

Rules of Thumb

● Business Intelligence is not Analytics

● Every aspect of a product is a hypothesis about what

users want

● Request feedback on your idea and product often

● Focus on short, simple feature releases

● Don't go live without analytics.

{"In God we trust; all other must bring data."} - W. Edwards Deming

The Mobile Ecosystem:

Integrated Apps:Standalone Apps:

Examples: Virtuoso v. ZipCar

APIs

Devices

Database

The Mobile Ecosystem: iOS & Android

Devices:● iPad(s)● iPhone(s)Languages:● Objective-C

Devices:● GalleryLanguages:● Java, XML,

SQLite

The Mobile Ecosystem: Cross Platform - Titanium & PhoneGap

Devices:● Android, iOS, Windows

Languages:● Javascript, CSS, HTML

The Mobile Ecosystem:

{'There can be significant performance differences on different technology platforms

and hardware that impact data gathering'}

{'There are limitations to the amount of campaign/marketing channel data that can be

collected depending on the tool one uses'}

The Mobile Ecosystem: Data Sources● Advertizing

○ iAd (iOS)○ AdMob (All)○ AdWords (All)

● In-app purchases○ iOS○ Android

● Link Sharing Channels○ LinkShare

● CRM Tools○ Mandrill○ Mail Chimp

● Services APIs○ Facebook○ Twitter○ Google○ Twilio○ Stripe

● Databases● Third Party

Analytics Solutions

Third Party Solutions

Our Focus:

List of Mobile Analytics Tools

Events v Pageviews

No Yes

Yes Yes

Pageviews Events

Google Analytics

Key Features:● It's free

● Many people already have a background using the tool

● Tracks the number and type of application crashes

● Campaign measurement parameters are pre-written

Drawbacks: ● "The Apple App Store does not currently support campaign measurement

using Google Analytics." (This does not apply to the Google Play Store)

● Not as customizable as other analytics tools

● There are issues with the existing plugins for cross-platform tools (Titanium, PhoneGap)

Google Analytics: Events v. Pageviews

MixPanel

Key Features:● Very customizable

● Standardized reports/dashboards are easy to setup and adjust

Drawbacks: ● Costs are incurred as collection increases

● Campaign measurement parameters must be defined programmatically

● Doesn't measure pageviews

The Analytics Lifecycle

So, what do I need to get started?1. An App project started

2. Choose & set up an analytics solution account

3. Install the analytics solution resources

4. Determine the parameters you're going to collect within

your application

5. Implement the data collection parameters throughout

your app

6. Test to ensure your parameters are collecting data

7. Start your analysis through standard reports &

dashboards

Development: The Mobile Development Cycle

Requirements & Design Build Test DeployAnalysis & Measurem

ent

Product Change Orders

Managing Product Changes

● Identify the reason for the change in user hypothesis with data

● Review your end to end functional & technical requirements and understand how the change will impact them

● Quantify the time and monetary cost of the change

Standard Reports & Dashboards

MixPanel: Google Analytics:

Data Points

Standard Reports:● Engagement:

○ Events, User Details, Search Terms, Funnels,

Trends, etc.

● Demographic Information:○ Country, City, Language

● Technology:○ Phone Model; Operating System; Network

Funnels

Funnels are extremely helpful in examining key user goals within the application

Formulas

Formulas allow you to establish Key Performance Indicators that can be tracked over time that are not standard reports

Dashboards & Bookmarks

Dashboards & Bookmarks allow for reports to be saved and even emailed out at a normal cadence

Google Analytics Campaign Parameters

*Reminder: Campaign measurement parameters will differ between app ecosystems

Social Media Integration

Facebook:● Build a page● Like, Share buttons

Twitter:● Choose your handle, hashtag● Tweet, follow buttons

Google+:● Build a page● + buttons

LinkedIn:● Build a page● Share, like buttons

{'Effective social media integration promotes branding, viral loops,

CRM, and is essential to consumer app growth.'}

Others:● Instagram ● Flickr ● Tumblr

Techniques & Tools: A/B Testing"A/B testing is a methodology in advertising of using randomized experiments with two variants, A and B, which are the control and treatment in the controlled experiment." - Wikipedia

A/B testing is designed to measure the effect of very small changes on user goals

Why is this different in Mobile than Web?:● Outcomes on websites can be programmatically updated realtime for users

● Adjustments to mobile apps need to be resubmitted

A/B Testing Example

https://www.lucidchart.com/documents/view/41ea-f704-515c7c3d-a817-22480a0045e5

Hypothesis:

An increase in the size of the logo on the sign in page of the application will cause a significant change in the amount of people who login to the application using the "Login with Facebook" button.

Measurement Tool:

MixPanel

Test Requirements:

Users opening the application will be assigned to either a user profile of Group A or Group B upon launching the application for the first time. They will then be randomly placed into either Group A or B and will navigate to screen 1.1.A or 1.1.B based on that assignment.

A/B Testing Example

https://www.lucidchart.com/documents/view/41ea-f704-515c7c3d-a817-22480a0045e5

Signin Screen of App:Group A:

Mixpanel *mixpanel = [Mixpanel sharedInstance];[mixpanel track:@"Signin App A"];

Group B:Mixpanel *mixpanel = [Mixpanel sharedInstance];[mixpanel track:@"Signin App B"];

Selection of Login w/Facebook:Group A:

Mixpanel *mixpanel = [Mixpanel sharedInstance];[mixpanel track:@"FB Click 1.1.A"];

Group B:Mixpanel *mixpanel = [Mixpanel sharedInstance];[mixpanel track:@"FB Click 1.1.B"];

A/B Testing Example

https://mixpanel.com/labs/split-test-calculator

Group A Group B

Signin to App 455 545

Login With Facebook 43 95

Click Through Rate (CTR) 9.45% 17.43%

Let's say we end up measuring this type of experiment over a month and we have 1000 app Signins over that period of time.

We know that the CTR is higher in Group B, but is it high enough for us to be confident in our test?

A/B Testing Example

https://mixpanel.com/labs/split-test-calculator

How do I take my skills expand my mobile analytics skillset?

Setup mobile analytics in your app!

Google Analytics Resources

MixPanel Education

Learn about common analytics experiments

Agenda Review

● Rules of Thumb

● The Mobile Ecosystem

● Third Party Solutions

● The Analytics Lifecycle

● Analytics in the Mobile Development

Process

● Standard Reports & Dashboards

● Statistical Techniques & Tools

APPENDIX

The App Ecosystem: Platform Choice

iOS Android Mobile Web

Examples:

The App Ecosystem: Platform Choice (Continued)

Examples:

PhoneGap Titanium

More: http://mashable.com/2012/03/28/mobile-app-development-advice/