webinar mobile conjoint analysis: are the results valid?

Gerard Loosschilder

CMethO

John Ashraf

Conjoint expert

Abigail Joffre

Today’s webinar host

#SKIMwebinar

Share your thoughts online:

Mobile Choice-based Conjoint

A valid alternative

http://www.twitter.com/skimgroup


Mobile surveys: it’s time to act

Increasingly people take surveys on a

mobile platform. Filtering them out

may skew the sample.

2

Source: Burke, July 2013

89%

4% 7%

2013

Smartphone

Tablet

PC

Screen

In

Out

Biased sample

Standard

survey design

Mobile survey

design

% of people taking

the test survey by

platform

SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”

But we should do a better job tailoring to their needs. The respondent may have something better to do

3

9%

10%

14%

24%

Mobile+

Mobile ST

Tablet

PC

Survey is enjoyable

31%

38%

53%

79%

Mobile+

Mobile ST

Tablet

PC

Responding was very easy

32%

35%

48%

66%

Mobile+

Mobile ST

Tablet

PC

Would take another survey on the same

device

Source: Burke, July 2013. Mobile ST is a standard survey on a smartphone; Mobile+ is a dedicated mobile survey design

This is

bad news

This is

bad news

This is

bad news


Introducing 3*3 mCBC

Today’s presentation is about the

validity of mobile choice-based conjoint

4 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”

Adapting to the shorter attention span of the mobile user

By reducing the number of choice tasks and number of choices per task

5

Three choice tasks Completed in

Three options each


At the same valid results as a traditional CBC exercise

The good news

The MAE of share predictions is

<4% at the correct sample size

Overall results, conclusions and

recommendations are the same

The bad news

So far, it is impossible to reduce

the number of choice tasks and

options per task to fewer than the

3*3 platform


3*3 mCBC requires a larger sample size

As a rule of thumb, sample size is

tied to number of parameters to be

estimated, determined by the

number of attributes and levels

Traditionally, we recommend a minimal

sample size of n= 200 for any conjoint

study. Now, the sample size goes up

with the number of parameters to be

estimated.

If # of

parameters is

Sample size

n=

12 (3 atts*4 levels) 400

16 (4 atts*4 levels) 800

20 (5 atts*4 levels) 1,200

25 (5 atts*5 levels) 1,600


3*3 mCBC requires a well-balanced research design

In a 3*3 mCBC research design,

the number of data points

collected is much smaller than the

number of parameters to be

estimated.

So, the key success factor is a

high-quality research design

Frequency balanced

Every combination of two

attributes and their levels is

represented well by all research

designs

Market knowledge included

We take current market shares

and knowledge about consumer

preferences into account in design

generation


How do we know if mCBC

using a 3*3 design is valid?

Empirical evidence


Because we have validated it …

We determined the relation between

sample size and convergent validity

It shows that at increasing sample sizes,

mobile CBC is a good approximation of

common CBC on traditional platforms


… in a study into pain killers and batteries

11

What is the validity of mobile CBC at various sample sizes

if compared with the same conjoint study in a traditional way at n= 200?

Attributes:

Brand

Price

Size

3-5 levels each;

12 parameters

Attributes:

Brand

Admin method

Dosage

Package

Price

3-5 levels each;

16 parameters


Attribute importance values show great similarities at all sample sizes for mobile CBC

12

The values for

batteries are

equally similar

Variations in

attribute

importance values

are insignificant

0%

10%

20%

30%

40%

Att

rib

ute

im

po

rtan

ce

Platform and sample size

Pain killers

Brand

Admin method

Dosage

Package

Price


However, at larger sample sizes, preference shares get closer to benchmark leg, demonstrating greater validity

13

Mean Absolute Error

(MAE) is a metric

expressing how far

apart the preferences

shares are as

measured by mCBC at

increasing sample

sizes compared with

the benchmark CBC

at n= 200 0.0%

2.5%

5.0%

7.5%

10.0%

200 300 400 500 600 700 800 900

MA

E v

alu

e i

n %

Sample size in the mobile leg

Mean Absolute Error values (MAE, in %)

Batteries

Pain killers

The break point is at n= 400; twice the

sample size of the benchmark

Especially at pain killers at 16

parameters (instead of 12 for batteries)


Conditions to make 3*3 mCBC work – research design

Every display of product profiles in a choice task is valuable. It

should not be wasted on prohibitions or alternative specific designs.

• We can not tolerate any reduction in D-efficiency values

So for now, we can apply mCBC in some standard situations.

We do not apply 3*3 mCBC in situations in which there are

prohibitions, utility balanced designs and alternative-specific

designs


Conditions to make 3*3 mCBC work – real estate

15

There is no space for large

amounts of information

Max. 5 attributes

Text is succinct, keywords

only

No complex descriptions

Pictures are

icons or schematic

Be careful with package or

ad testing


16

Interaction design

http://tinyurl.com/skimmcbc

Powered by


3*3 mCBC in Price Sensitivity Analyses – batteries

17



3*3 mCBC in Price Sensitivity Analyses – pain killers

18



3*3 mCBC in Telecom – design of a postpaid portfolio

19



Mobile in MaxDiff applied to Semantics and other tests of claims and benefit statements

20



3*3 mCBC is a valid addition

To conclude


To conclude – mCBC is a valid addition

Do

• Take market knowledge into

account

• Use a larger sample

• Use succinct text, iconized

visuals

• Have five attributes at five

levels max

Don’t

• Use it for large CBCs

• Include research design

restrictions

• Use it for package, ad or

concept work


Next steps

• SKIM researchers will continue to validate mCBC …

• For allocation studies

• For large research designs using consideration sets

• For more complex visuals

• Keep improving UI for Mobile CBCs

• Across dominant platforms

• Translate learnings back to any CBC


Go to www.skimgroup.com/webinars

for today’s presentation slides and more!

John Ashraf Conjoint expert

[email protected]

Abigail Joffre [email protected]

+1 201 963 8430

@SKIMgroup

Gerard Loosschilder Chief Methodology Officer

[email protected]

@gloosschilder

#SKIMwebinar

Share your thoughts online:

http://www.skimgroup.com/webinars

mailto:[email protected]





http://www.facebook.com/skimgroup


http://www.linkedin.com/company/skim

http://www.youtube.com/skimvideos

webinar mobile conjoint analysis: are the results valid?

Business