Deutsche Telekom AG – Marketing Communications Strategy & Media
in Cooperation with SKOPOS GmbH & Co. KG
March 2014
New Enabling of Advertising Research Using Facial Coding and Real Time Response Measurement Online Dr. Olaf Wenzel (SKOPOS) Christian Lauterbach (Deutsche Telekom)
And what do you think about this sequence? Tell me about your feelings watching this.
WHAT‘S RIGHT WITH OUR TV-SPOT? IS THERE SOMETHING WRONG? CLASSIC RESEARCH IS NEEDS TO ASK QUESTIONS OVER QUESTIONS ...
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
2 March 2014
And what do you think about this sequence? Tell me about your feelings watching this.
And what do you think about this sequence? Tell me about your feelings watching this.
So we are asking open questions to get a deeper understanding of the performance of a TV-Spot.
An overall rating is important but it does not tell us the whole story.
Finally we have to conduct content analysis to grasp which sequence works well, and why that is so.
CAN FACIAL CODING OR REAL TIME RESPONSE MEASURM. HELP US? WHAT ARE THE BENEFITS? WHAT ARE THE PITFALLS?
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
3 March 2014
REAL TIME RESPONSE MEASUREMENT
Test person is rating the stimulus while watching it using the mouse wheel:
• Wheel up = like
• Wheel down = don‘t like
TECHNICAL REQUIREMENTS:
• Wheelmouse
• Software plugin hosted on an external server
RESULT:
One explicit measure over time.
AUTOMATED FACIAL CODING
Test person is watching the stimulus. The testperson‘s face is captured on video using its webcam.
TECHNICAL REQUIREMENTS:
• Webcam
• Software plugin hosted on an external server.
RESULTS:
Seven implicit measures decoded from facial expressions
• Emotion lift (combined overall measure)
• Anger, Disgust, Happiness Fear, Sadness, Surprise
Source: http://w
ww
.nviso.ch/technology.html
LET‘S GIVE IT A TRY ... TESTING TWO TV-SPOTS USING THREE DIFFERENT APPROACHES.
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
4 March 2014
1.
2.
3.
A. CLASSIC APPROACH
Presenting the 1st video
Asking questions, rating the video
Presenting the 2nd video
Asking questions and rating the video
B. FACIAL CODING (FC)
1. Presenting the 1st video and measuring emotions via facial coding
2. Presenting the 2nd video and measuring emotions via facial coding
3. Presenting the 1st video again
4. Asking questions, rating the video
5. Presenting the 2nd video again
6. Asking questions and rating the video
C. REAL TIME RESPONSE MEASUREMENT (RTRM)
1. Presenting the 1st video and rating via RTRM
2. Presenting the 2nd video and rating via RTRM
3. Presenting the 1st video again
4. Asking questions, rating the video
5. Presenting the 2nd video again
6. Asking questions and rating the video
RA
ND
OM
OR
DE
R
AT A FIRST GLANCE THE OUTCOMES SEEM TO BE QUITE SIMILAR. FC AND RTRM RATING CURVES SHOW QUITE SIMILAR GRADIENTS.
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
5 March 2014
RTRM
Facial Coding
RTRM
Facial Coding
* FACIAL CODING CURVES ARE REPRESENTING EMOTION LIFT.
Emotion lift is a combined measure of the six basic emotions measured via facial coding: anger, disgust, fear, surprise, happiness, sadness.
RTRM
Facial Coding
RTRM
Facial Coding
AT A FIRST GLANCE THE OUTCOMES SEEM TO BE QUITE SIMILAR. FC AND RTRM RATING CURVES SHOW QUITE SIMILAR GRADIENTS.
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
6 March 2014
* FACIAL CODING CURVES ARE REPRESENTING EMOTION LIFT.
Emotion lift is a combined measure of the six basic emotions measured via facial coding: anger, disgust, fear, surprise, happiness, sadness.
FC REACTS FASTER ON CHANGES OF EMOTIONAL STATE. RTRM SEEMS TO SUFFER FROM SOME KIND OF LATENCY.
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
7 March 2014
RTRM
Facial Coding
FACIAL CODING (EMOTION LIFT) VS. REAL TIME RESPONSE MEASUREMENT (SPOT 2)
Whilst FC shows instant reactions to
the stimulus …
Probands need to think first about what they see before
they react via RTRM. … RTRM occasionaly seems
to lack a little behind.
WHAT MAKES YOU HAPPY? WHAT IS MAKING YOU SURPRISE? SOME FC MEASURES ARE CORRESPONDING WELL WITH THE CONTENT.
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
8 March 2014
HAPPINESS CURVE (SPOT 1 VS. SPOT 2) / FACIAL CODING
Happiness goes quickly down when
construction worker is on display.
Happiness goes only slightly down when the
last scene is on display.
WHAT MAKES YOU HAPPY? WHAT IS MAKING YOU SURPRISE? SOME FC MEASURES ARE CORRESPONDING WELL WITH THE CONTENT.
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
9 March 2014
SURPRISE CURVE (SPOT 2) / FACIAL CODING
When (all of a sudden) the city is emerging the surprise measure goes
straight up.
Once the lines have been identified as
magenta colored veins the surprise measure
goes down.
RTRM MEASURES EXPLICIT. FC MEASURES IMPLICIT. HIGH CORRELATION OF RTRM RESULTS WITH RATING SCALES.
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
10 March 2014
0,14
-0,03 -0,03
0,10 0,04
-0,06 -0,13
0,48
0,11
-0,07
0,09 0,08
-0,16
0,05
-0,01
0,63
Emotion lift Surprise Sadness Happiness Fear Disgust Anger RTRM
Spot 1 Spot 2
CORRELATIONS OF AGGREGATED MEASURES WITH OVERALL RATING OF THE SPOT (FC vs. RTRM)
RTRM measure correlates highly significant with
explicit overall ratings of the spots.
FC measures show no substantial correlations with explicit overall ratings ot the
spot.
FACIAL CODING MEASURES
MEASURES ARE OF USE ONLY IF PREASSUMPTIONS ARE PROFOUND. INTERPRETATION SHOULD CONSIDER COMMUNICATION TARGETS.
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
11 March 2014
ANGER
DISGUST
SURPRISE
HAPPINESS
FEAR
SADNESS
EMOTION LIFT
?
?
?
INTERPRET WITH CAUTION!
Most of the measures can not be claimed to be „good“ or „bad“ per se.
CONSIDER COMMUNICATION TARGETS!
Examples
Target: Perceive company (sender) in a happy mood!
Focus on happiness level before logo is displayed
Target: Induce activation at the beginning of the spot!
Focus on surprise, disgust, fear at the beginning
REMEMBER THAT THE TEST SITUATION IS OUT OF YOUR CONTROL. IN 24% OF ALL CASES FC WAS NOT WORKING FOR TECHNICAL REASONS.
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
12 March 2014
FACIAL CODING WORKS WELL FACIAL CODING MIGHT WORK FACIAL CODING NOT POSSIBLE
SOME SCREENSHOTS OF TEST PERSONS (AVAILABLE FOR QUALITY CONTROL)
REMEMBER THAT THE TEST SITUATION IS OUT OF YOUR CONTROL. IN 27% OF THE CASES OF RTRM THE MOUSE WHEEL WAS NOT USED.
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
13 March 2014
73% of the respondents are really using their mouse wheel to rate the stimulus.
27% of the respondents are doing nothing but watching („Flatliners“)
DIFFERENT GROUPS OF RTRM RESPONDENTS
FC PROVIDES A CLEAR BENEFIT. RTRM NOT THAT MUCH. ONLINE TESTING WORKS. BUT OVERSAMPLING IS REQUIRED.
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
14 March 2014
RTRM suffers from latency. It is not quite clear (and variing) how fast a decision is made about liking or not liking a scene. The measure is not precise regarding the timeline. We end up with an unprecise explicit measure not carring much more information than the classic approach via asking questions.
Facial coding has some advantages.
• Precise in terms of measuring spontaneous reactions implicitely. No latency enforced by cognitive appraisal.
• Adding value in terms of adding new qualitative information going beyond common explicit measures.
• Speeding up the research process by making long and repeated questions refering to different scenes obsolete.
But we don‘t know yet which level of emotion lift or of the other measures has to be achieved in terms of being really substantial according to our communication tagets. Benchmarking is required!
From a methodological point of view:
• RTRM and FC both are valid options for online research. Face validity and criterion-related validity are high.
• But substantial oversampling is required for both methods due to technical reasons and due to the lack of control of the interview situation.
THANK YOU!
Deutsche Telekom AG Marketing Communications Strategy & Media in cooperation with SKOPOS GmbH & Co. KG
15 March 2014
Christian Lauterbach
Senior Expert Communications
Deutsche Telekom AG
Dr. Olaf Wenzel
Research Director
SKOPOS GmbH & Co. KG