giving organisations new capabilities to ask the right business questions 1.7
DESCRIPTION
This presentation takes the seminal work structured analytic techniques work pioneered within US intelligence, and proposes adaptions and simplifications for use within commercial enterprisesTRANSCRIPT
Giving Organisations
new Capabilities
to ask the Right
Business Questions
Stephen Simpson
@sharplyunclear
Value Captured
Sales Growth Profit Growth
Sales Growth to Existing
CustomersNew Customers
Product PerformanceTechnology
Leadership
Process
Improvement
Effective Project Execution
Balanced Innovation
Portfolio
Quality of Innovation
PipelinePartners’
Value-add
Employee
Commitment to
Innovation
Supportive Strategy,
Structure, & Systems Access to Talent
Outputs
Outcomes
Processes
Inputs
Making Data Work is Hard
Myron Ullman, CEO
Ron Johnson, CEO
The “All In” Approach
You start out thinking you have a sales problem but might find
it is not really sales but marketing or customer retention...
…you could spent a lot of time on analysis that doesn’t lead to
solving the right problem.”
"We did a Hadoop trial last year, it didn't go very far
because we weren't getting the intelligence out of it
that we thought we would. So we are looking at some
other initiatives with different vendors this year.
"We tried to put three different data sets together, and
then tried to see if we could find some causality
between the data sets that would gives us intelligence
that would allow us to manage our operations better…
"Whether that was how we set the trial up or the
software I don't know, so we are going to try
some different things.”
The Experimental Approach
Incumbents are rarely disrupted by new technologies they can't
catch up to, but instead by new business models they can't match.
Institutions will try to preserve the problem to which they are
the solution.
The “Wait and See” Approach
Can rarely evaluate all outcomes with sufficient precision
Usually don’t know relevant probabilities of outcomes
Possess limited memory
Satisficing
Results are often Modest
“All other things being equal”
When we sacrifice dealing with detail complexity to focus on dynamic
complexity, the solutions don’t produce the outcomes that we really want.http://blogs.hbr.org/2013/09/our-self-inflicted-complexity/
Business
StrategyDomain
Expertise
Data Mining
Company Systems & Data
Sourcing
Extraction
Interpretation
Visualisation
Implementation
Agile
Experimentation
We need to make
sure that we’re
asking people to
research the right
questions
We need to look in
many more places to
find data…
…and it will take a lot of
different skills and
approaches to bring it
together
We need to be careful to
curb our enthusiasm and
separate out the signal
from the noise
We need new techniques to
interpret and manipulate vast
numbers of data points on a
single surface
We need simple, easy to
use production tools to act
upon the new insights.
Authority needs to be
delegated to where the
information is captured
We need to perform analysis
quickly inside small projects,
with a specific business goal.
Some of these will fail.
And then we iterate to improve the
insight gained, or address the next
business question…
We need to choose the
right storage technologies,
integration services &
architecture
Business
Strategy
Obtaining new insights
CRISP-DM
Candidate Sources
Richards J. Heuer &
Randolph H. Pherson
Structured Analysis
Expert Judgment
Quantitative
Methods using
Expert-Generated
Data
Quantitative
Methods using
Empirical Data
Decomposition &
Visualisation
Idea Generation
Scenarios & Indicators
Hypothesis Generation
& Testing
Assessment of
Cause & Effect
Challenge Analysis
Conflict Management
Decision SupportStructured Analysis
Analytic Methods
14
a step by step process for analyzing the kind of
incomplete, ambiguous and sometimes deceptive
information that analysts must deal with.
Structured Analysis
16
Diagnostic + Contrarian + Imagination elements
Structured Analytic Techniques contain
The Techniques
1. Define the
project?2. Get started?
3. Examine & make
sense of the data?
Figure out what is
going on?
4. Assess the most
likely outcome of an
evolving situation?
5. Monitor a
situation to avoid
surprise?
9. Challenge your
own mental model?
6. Generate and test
hypotheses?
7. Assess the
possibility of
deception?
8. Foresee the
future?
10. See events from
the perspective of
other players?
11. Managing
conflicting mental
models or
opinions?
12. Support a
manager in
deciding course of
action?
Decomposition & Visualisation
Idea Generation
Scenarios & Indicators
Hypothesis Generation
& Testing
Assessment of
Cause & Effect
Challenge Analysis
Conflict Management
Decision Support
1. Define the
project?
Choosing what you want to doDecomposition & Visualisation
Idea Generation
8. Foresee the
future?
Scenarios & Indicators
Hypothesis Generation
& Testing
Assessment of
Cause & Effect
Challenge Analysis
Decision Support
6. Generate and test
hypotheses?Hypothesis Generation
& Testing
Assessment of
Cause & Effect
12. Support a
manager in
deciding course of
action?
Hypothesis Generation
& Testing
Conflict Management
Decision Support
Template Structure
20
Overview
When to Use It
Value Added
The Method
Relationship to other Techniques
Origins of this Technique
21
Personalised Interactions
Long term
personal healthcare
Branded Currency
When forced to work within a strict framework the imagination
is taxed to its utmost – and will produce its richest ideas.
Given freedom the work is likely to sprawl.
1. Decomposition & Visualisation
Client micro-segmentation using multiple sources of data
Scenarios
Description
Understand your clients’ needs at the finest level of detail
FOR marketing operations
WHO want to understand the growth potential for each identified customer subdivision
THE understand your clients’ needs at the finest level of detail solution
PROVIDES understanding of the root causes for your current share of each identified slice
THAT lets you act on the information quickly with targeted retail product placement & location selling
UNLIKE your existing solution
WHICH is coarse-grained and retrospective
• Retail product placement & location selling
• Counteracting effectiveness of competitors
• Understanding local reputation via ”voice of the customer”
• Real-time decision making such as mobile-based coupon positioning to particular segments
• Partner organisations’ service effectiveness
Value Proposition
The best way to have a good idea is to have a lot of ideas
2. Idea Generation
Out-of-box thinking
Raw & refined ideas
Experimentation
Ambiguity/uncertainty
Research
Intuition
Surprise
Courage
Find the right things
Ask questions & explore
unknown innovation
Seize opportunities
Visualize future & consider
all options
Include incremental &
radical ideas
In-the-box thinking
Engineering/process
improvement
Precision
Well-calculated trade-offs
Buying/selling of ideas
Do things right
Answer questions & verify
solutions
Avoid major risks
Get product into the
marketplace
Bias for incremental
Creativity Value Creation
Cross Impact Matrix
For when “Everything is connected to everything else”
Business is in flux
System is stable
- Need to identify and monitor all
factors that might upset this
A significant event has occurred
- Need to understand implications
Context for discussion of interactions
Discover variables once thought to be simple
& independent are actually interrelated
Focus on
- Interactions that may have been overlooked
- Variables that might reinforce each other
A B C D E F
A. Personalised Interactions ++ ++
B. Existing mobile solutions --
C. Existing core banking solutions -- ++ -
D. Apps & Cloud Service interaction + - + +
E. Offers ++ - ++ ++
F. Analytics ++ ++
Cross Impact Matrix
3. Scenarios & Indicators
Scenarios are plausible &
provocative stories about how
the future might unfold
Observable Phenomena that can periodically be reviewed to help track events
Make humans recognize early signs
significant change
Spot emerging trends
- Warn unanticipated changes
- Avoid surprise
Forward looking, predictive
Objective baseline for tracking
Instil rigour into analytic process
Enhance credibility of what delivered
Exchange knowledge between experts
from different domains
Quality indicators are critical
- If narrowly defined or out of date
- Reinforce bias
- Discard new evidence
- Lull people inappropriately
- Dashboards…
Indicators Validator
- Quality and strength of indicator
- Whether appears in all scenarios
Indicators
Indicators
Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
Mobile Offers
Reaching right segment
People engaged
Volunteering information
Infrastructure
Holding initiative back
Cloud
Security, regulatory, compliance
Service
Take-up standard services
3rd party composing new apps
Industry Trends
Personalised CRM
Branded Currency
Device as Bank
Ecosystem
Retailers using your backbone
Competitive launches
Strong
Substantial
Moderate
Low concern
Neglible concern
201520142013
□
●○
▪▫
▫
▫
▫▫
□○
□
●□
●
●
□□
□
●
▫▫
▪
□
○○
●
▪□
□●
A possible explanation of the past or a judgment about
the future is a hypothesis that needs to be tested by
collecting and presenting evidence
4. Hypothesis Generation & Testing
We are slow to accept the reality
of simple mistakes, accidents,
unintended consequences,
coincidences, or small causes
leading to large effects
5. Assessment of Cause & Effect
Personalised Interactions will increase: Key Assumptions check
Legal and privacy – Caveated.
Components available across entire chain – Caveated.
Customers want seamless, personally relevant services – Solid
Devices will progress sufficiently – Solid
Analytics techniques are sufficiently refined, accurate and timely – Caveated
Back-end systems will support workload – Solid
Systems will be cost effective – Caveated. What’s the ROI of something you don’t know?
Employees trained and authority delegated to act – Unsupported
It is the mark of an educated mind to be able to entertain a
thought without accepting it.
6. Challenge Analysis
Imagine the future where your plan has been implemented, but has failed
Advantages:
Take people out of perspective of
defending their plan & shielding
themselves from its flaws
Increase level of candour
Can be used to show decision makers
that are typically over-confident that
their decisions and plans will work
Questions re-framed, to elicit different
responses to original ones
Legitimises dissent – asked to make a
positive contribution by identifying
weaknesses in previous analysis
Pre-mortem analysis
Examples
- Internal inertia or uneven execution
- Competitors’ actions
- Law of unintended consequences
- Economic changes
7. Conflict Management
Disagreements sparked by differences in perspective, competencies,
& access to information… actually generate much of the value that can
come from collaboration across organisational boundaries.http://hbr.org/2005/03/want-collaboration-accept-and-actively-manage-conflict
…without overstepping the limits of their role…; just structures all the
relevant information in a format that makes it easier for the decision
maker to make a choice.
8. Decision Support
A word on Dashboards
It is also unfortunate to see how many business intelligence
and enterprise data warehousing projects get waylaid by the
singular pursuit of pretty dashboards…
Iterating Quickly
Time is Key
Does provide new capabilities to ask right questions
- Offers path to clearer business goals
- Discourages “wait and see” approaches
Encourages cross-organisational linkages
Validates or challenges experts’ “hunches”
More limited use in monitoring subsequent change
12
In Summary