teaching computers to think like decision makers: the next revolution in the data sciences
DESCRIPTION
“Big Data” and analytics have revolutionized "micro-decisions", those myriads of tiny decisions that follow a similar pattern, are made frequently, but each of which has relatively low risk and low value (e.g. the cross-sell to “things we might also like” that almost every e-commerce checkout page displays using our purchase history and possibly other data). By contrast, "macro-decisions" are less frequent, but higher-stakes. They are more complex and also need to take risk into account. Software support for macro decisions today is usually provided as “Business Intelligence” or “Dashboards”, both of which typically derive aggregate statistics from existing data, and present these in ways that are “meaningful” and “insightful” to humans. However, once the data has been presented, the synthesis and evaluation tasks at the core of the decision-making process are left to the human decision-maker. This is despite a large and well-accepted body of research (most notably by Kahneman and Tversky) clearly demonstrating that humans systematically lack the ability to perform such tasks accurately. A significant and as-yet untapped opportunity therefore exists for augmenting the existing BI paradigm with new data science techniques developed to assist decision makers. This presentation introduces the “Decision Intelligence” approach which transfers the decision-related inference tasks from human intelligence to machine intelligence. The approach includes a structured framework for decomposing decisions so they can be represented as computable models. Using simulation and optimization techniques, these models generate data sets to which existing BI tools can be applied, giving decision makers the ability to generate data from “possible futures” and to evaluate decision and their outcomes in familiar, existing environments. Mark is a leader in innovative research, software development and services delivery, and business development in the academic and commercial sectors for over two decades. He is co-founder and CEO of Quantellia, a leading Data Science innovator and developer of the award-winning World Modeler software. From 2000-2010, he held the position of CTO at Spatial info (now Synchronoss) where he co-founded the company’s US operations, and led technical operations. Prior to this, he was the architect of StatPlay, software developed jointly at La Trobe University and the University of Melbourne that explored how computer visualizations affect people’s innate abilities to perform statistical reasoning. Mark has also worked as a systems engineer for EDS (now HP) and Anderson Consulting (now Accenture). In 1994-5, he held a British Council Post Graduate Bursary at the University of Cambridge in the UK and from 1996-2000 was an Honorary Fellow at the University of Melbourne. He is the author of numerous publications and has frequently made speaking appearances.TRANSCRIPT
Teaching Computers to Think Like Decision Makers
Mark ZangariCEO, Quantellia LLC
San Francisco UniversityMay 23, 2014
[email protected] 717 4221
Copyright © 2014 Quantellia LLC. All Rights Reserved.
Robert McNamara• Secretary of Defense (1961-68)• Ford Motor Co. (1946-61)• USAF “Statistical Control” (1943-46)
Data System Analysis
Decision
http://sunsite.berkeley.edu/FindingAids/dynaweb/calher/jvac/figures/j12EB-644A.jpg
http://www.whatswrongwiththeworld.net/office-interior-1940s.jpg
http://www.biega.com/bcbphotos/biega-engineer.jpghttp://lcweb2.loc.gov/service/pnp/hec/28300/28336r.jpg
Data Acquisition…
Data Storage…
Data Mining…
Analytics…
Data
http://sunsite.berkeley.edu/FindingAids/dynaweb/calher/jvac/figures/j12EB-644A.jpg
http://www.whatswrongwiththeworld.net/office-interior-1940s.jpg
http://www.biega.com/bcbphotos/biega-engineer.jpghttp://lcweb2.loc.gov/service/pnp/hec/28300/28336r.jpg
Data Acquisition… Data Mining…
Analytics…
Data
Data Storage…
DataInstrumented
Code / Sensors
DataManagement
Analytics
Presentation
System Analysis
Decision
DataInstrumented
Code / Sensors
DataManagement
Analytics
Presentation
Big Data
Business
Intelligence
Demarcation between automated (computer-centric) and manual (human-centric) information processing.
Gap between computer and humanbridged by Data Visualization.
Units Cost Per Unit1-100 $12.00
101-500 $10.00
501-1000 $9.00
1001-10000 $7.50
10001+ $6.00
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Sale
s Vol
ume
/ M
arke
t Siz
e
Retail Price
Base Demand
0.00%
1.00%
2.00%
3.00%
4.00%
5.00%
6.00%
7.00%
8.00%
9.00%
$0$5
00$1
,000
$1,5
00$2
,000
$2,5
00$3
,000
$3,5
00$4
,000
$4,5
00$5
,000
$5,5
00$6
,000
$6,5
00$7
,000
$7,5
00$8
,000
$8,5
00$9
,000
$9,5
00$1
0,00
0$1
0,50
0$1
1,00
0$1
1,50
0$1
1,75
0
Pct.
Incr
ease
in D
eman
dMarketing Spend
Marketing Driven Demand Uplift
Manufacturing Unit Cost by Volume
The Product Manager’s Decision:
To maximize profit…a) How many units do I order from the
manufacturer?b) What retail price do I charge?c) How much of my profit do I re-invest
in marketing?
(Mkt Size = 50,000)
Even with all the data you need, and clear visualizations, making good
decisions is still very hard to do.Why?
Data System Analysis Decisionü
Because:
a) Humans are not good at runningSystems in their heads.
b) Unlike Data, there is little mainstream computerized support for modeling and analyzing Systems.
(But let’s see if we can change that…)
Build a Computable Systems Model Visually
• Attributes• Dependencies
The Product Manager’s Model and quickly
Identify Model Elements:
• Outcomes / Goals“What are we trying to achieve?”
• Levers“What can we control?”
• Externals“What affects our outcomesthat we can’t control?”
Build a Computable Systems Model Visually
Identify DependenciesDependencies“How are A, B and C related to X, Y and Z?”IntermediatesWhen outcomes are not directly related to levers or externals.
Build a Computable Systems Model Visually
Quantify DependenciesDependencies“How are A, B and C related to X, Y and Z?”
Build a Computable Systems Model Visually
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Sale
s Vol
ume
/ Mar
ket S
ize
Retail Price
Base Demand
Expressions
External Data Sources / AnalyticsSketch Graphs
Quantify Dependencies
Dependencies“How are A, B and C related to X, Y and Z?”
Build a Computable Systems Model Visually
Models also provide a systematic way to assess the impact of uncertainty, sensitivity, precision and risk on the decisions they support.
While humans are not good at processing systems models, we are much better at analyzing and designing them. This leads to a natural human-computer partnership.
Build a Computable Systems Model Visually
The Product Manager’s Decision:
a) How many units do I order from the manufacturer?
b) What retail price to I charge?c) How much of my profit do I
re-invest in marketing?
… to maximize profit?
But wait, there’s more.
38,000$15
7%
The Product Manager’s Decision:
Most decisions are made not justto optimize outcomes, but to managerisk.
A bi-product of the optimization search is data that can be used to:• Assess sensitivity of the desired
outcome to particular levers and externals.
• Assess downside risk associated with each positive outcome.
Opportunity envelopeRisk envelope
Gradient shows sensitivity
Some Interesting Structural Characteristics of Models…Build a Computable Systems Model Visually
Feedback Loop
… Lead to Important Behaviors.Equilibrium and Transient States
• Real-life systems, even if they are stable, are not static, but in a steady state or equilibrium.
• When such systems are perturbed, they oscillate, or experience a transient.
• Effective decision makers need to be able to understand the effects their decisions will have both on the transient phase and on the new equilibrium.
Build a Computable Systems Model Visually
Equilibrium with price at $12
Price raisedto $15
New equilibrium with price at $15
Transientphase
Data System Analysis
Decision
Big Data / Business Intelligence:
Data
System Analysis
Decision
Decision Intelligence
Analyze system
Build model
Integrate Data to specify dependencies
Search the space of decision leversand externals to determine
optimal outcomes and risk profiles
Gap between computer and humanbridged by Data Visualization of Decision Variables, not the Input Variables as before.
Decision Intelligence:
• Gives decision makers what they need most, and they cannot get from Business Intelligence: help answering the question “If I make this decision, then what will be the likely results, and what risks am I exposed to?”
• Provides a framework for the most effective use of existing data and analytics tools in a given problem.
• Provides visual and other artifacts that assure team alignment and act as a form of “institutional memory”
New Kinds of Visualizations
• Familiar data visualizations still have their place in Decision Intelligence, but note that the “axes” are now more meaningful to decision makers as each represents an “actionable” quantity.
• In addition, there is a powerful role for new dynamic System Visualizations.
Call to Action:
Now that the “Big Data” problem is mostly solved, we need invest our talents to return to the “Big
Picture”.
We must develop software tools and methodologies that integrate data and systems to
produce the kinds of insights real users really need.
Download a free trial of World Modeler from www.quantellia.com
Mark ZangariCEO, Quantellia LLC
San Francisco UniversityMay 23, 2014
[email protected] 717 4221
Copyright © 2014 Quantellia LLC. All Rights Reserved.
Thank You.