the future of advance analytics
DESCRIPTION
Presented on April 17th for InnoTech Dallas.TRANSCRIPT
4/1/2013
1
The Future of Advance Analytics
David SmithChief Executive Officer, HBMG [email protected]:InnoTech Main Stage Theater, inside Hall Bg ,
Volume, variety, and velocity are changing us in our companies, government agencies and at home. How do BI, Social/Business media, mobility, Devices and Big Data drive business decisions? The success is driven by the use of advance analytics. Business analytics facilitates realization of business objectives through reporting of data to analyze trends, creating predictive models for forecasting and optimizing business processes for enhanced performance. This is important not only in business but the military and government as well.
This presentation will look at the current and future trends in analytics and how they will impact each of us. Special emphasis will be given to the new trend of embedded analytics. As the world moves faster toward real-time the analytics must move as well. Attendees will leave the session with a understanding of the future directions of advance analytics.
The Future of Advance AnalyticsAnalytics
David SmithCEO [email protected]
4/1/2013
2
Definition for Advanced Analytics
Analysis is the examination process itself where analytics is the supporting technology and associated tools. BI is quite synonymous to analytics in IT context. Advanced Analytics, Business Analytics, Data Analytics, Analytics Software, Analytics Technology are almost always marketing pleonasms (redundantalways marketing pleonasms (redundant expressions) and can be safely substituted by just ‘analytics’
Definition for Advanced Analytics
Analysis is a pretty old, well understood term d ti ll “b ki d ”and essentially means “breaking down” or
“decomposition”. More accurately –the process of decomposing complex entity into simpler components for easier comprehension.
4/1/2013
3
Definition for Advanced Analytics
Advanced analytics provides algorithms for complex analysis of either structured or unstructured data. It includes sophisticated statistical models, machine learning, neural networks, text analytics, and other advanced data mining techniques. Among its many use cases, it can be deployed to find patterns in data, prediction, optimization, forecasting, and for complex event processing/analysis. Examples include predicting churn, identifying fraud, market basket analysis, or understanding website behavior Advanced analytics does notunderstanding website behavior. Advanced analytics does not include database query and reporting and OLAP cubes.
America gets more than half its economic growth from industries that barely existed
a decade ago—such is the power of innovation, especially in the information
and biotechnology industries.
—The Economist
Copyright, 2011 © HBMG, Inc.
4/1/2013
4
Business Problem
More than half of business and IT executives, 56 percent, report they feel overwhelmed by the amount of data their company manages. Many report they are often delayed in making important decisions as a result of too much information. Surprisingly, 62 percent of C‐level respondents – whose time is considered the respondents whose time is considered themost valuable in most organizations – report being frequently interrupted by irrelevant incoming data.
4/1/2013
5
Delivering business value is hard…
• “Of the work executed: “Many (possibly most) organizations lose as much as 45% of their total revenues due to costs associated with low quality”associated with low quality– Six Sigma
• “Some 75 percent of most large‐scale J2EE projects fail by missing both time and budget projections …”– Mark Driver, Gartner
• “64% of features actually delivered are• 64% of features actually delivered are either rarely or never used” – Jim Johnson, Standish Group
Why?
• Technological innovation is now the most important driver for competitive success
– Many firms earn over one‐third of sales on products developed within last five years
• Product life cycles ( time between product introduction to market and its withdrawal)
Copyright, 2011 © HBMG, Inc.
– Software 4‐12 months
– Computer hardware 12‐24 months
– Large home appliances 18‐36 months
4/1/2013
6
Business, Knowledge, and Innovation Landscape
• Typically 80% of the key knowledge (and value) is held by 20% of the people – we need to get it to theheld by 20% of the people we need to get it to the right people
• Only 20% of the knowledge in an organization is typically used (the rest being undiscovered or under‐utilized)
80 90% f th d t d i t d ill b• 80‐90% of the products and services today will be obsolete in 10 years – companies need to innovate & invent faster
Copyright 2012@ HBMG Inc.
Tapping into the Data
• Data Storage• Reporting• Analytics
Utilized data• Analytics• Advanced
Analytics
– Computing with big datasets is a
Unutilized data that can be
il bl tg
fundamentally different challenge than doing “big compute” over a small dataset
available to business
4/1/2013
7
Innovation:‘The real voyage of discovery consists
not in seeing new lands, but in seeing with new eyes’
(Source: Marcel Proust)
‘Innovation = creative idea and implementation’
(Source: Glossary of Electronics)
‘Innovation: change that
‘A new method, idea, product, etc’
(Source: Oxford English Dictionary)
creates a new dimension of performance’
(Source: Peter Drucker)
‘An innovation to be effective has to be simple and it has
to be focused’(Source: Peter Drucker)
‘Value innovators look for what customers value in
common’(Source: Kim & Mauborgne)
‘Innovation is the process by which new products or methods of production are introduced, including all the steps from the inventor’s idea to bringing the
new item to market’(Source: Baumol, Economics: Principles & Policy)
(Source: Peter Drucker)
‘Firms need to manage steady state innovation and radical change because continuous improvement is no longer
enough’(Source: Tom Peters)
“Big Data” and it’s close relatives “Cloud Computing”, “Social Media” and
"Mobile"
are the new frontier of innovation.
Driven by Advance Analytics
4/1/2013
8
Big Data and It’s Brothers
VolumeVolume
Variety
Velocity
………..
Volume
Volume is increasing at incredible rates. h l h h dWith more people using high speed
internet connections than ever, plus these people becoming more proficient at creating content and just more people in general contributing information are combined forces thatinformation are combined forces that are causing this tremendous increase in Volume.
4/1/2013
9
Variety
Next in breaking down Big Data into easily digestible bite‐size chunks is the concept of Variety. Take your personal experience and think about how much information youexperience and think about how much information you create and contribute in your daily routine. Your voicemails, your e‐mails, your file shares, your TV viewing habits, your Facebook updates, your LinkedIn activity, your credit card transactions, etc.
Whether you consciously think about it or not the VarietyWhether you consciously think about it or not the Variety of information you personally create on a daily basis which is being collected and analyzed is simply overwhelming.
Velocity
The speed at which data enters organizations these days is absolutely amazing. With mega internet bandwidth nearly being common place anymore in conjunction with the proliferation of mobile devices, this simply gives people more opportunity than ever to contribute content to storage systems. contribute content to storage systems.
4/1/2013
10
Worldwide digital content will double in 18 months, and every 18 months thereafter.
VELOCITY
IDC
CRM Data
GP
S
Demand
Sp
ee
d
VelocityO
ppor
tuni
ties
Customer
Inventory
Em
ails
Tw
eets
Planning
Mobile
Instant MessagesVOLUME VARIETY
Transactions
Se
rvice C
alls
Sales Orders
ThingsIn 2005, humankind created 150 exabytes of information. In 2011, over 1,200 exabytes was created.
VOLUME80% of enterprise data will be unstructured, spanning traditional and non traditional sources.
Gartner
The Economist
But I Believe there are Four V4
4/1/2013
11
• Volume:Gigabyte(109), Terabyte(1012), Petabyte(1015), Exabyte(1018), Zettabytes(1021)
• Variety: Structured,semi‐structured, unstructured; Text, image, audio, video, record
• Velocity(Dynamic, sometimes time‐varying)
• BUT needs to add and create Value!• BUT needs to add and create Value!
Trends driving data management
– The volume of data has never been greater and is growing exponentially
– The value of data has never been better understood– The value of data has never been better understood
– The capabilities for processing data have never been better
• Higher processor performance and density are enabling advanced processing on commodity hardware
• Software enhancements designed to make best use of i f d l bl hit tprocessing performance and scalable architecture
• Advanced and in‐database analytics bring processing to the data, reducing latency and improving efficiency
– The data deluge problem is also a big data opportunity
22
4/1/2013
12
Copyright, 2011 © HBMG, Inc.From http://geekandpoke.typepad.com
Next Generation
• Cloud Computing (e.g. “Blue Cloud”)
– The “network becomes the computer”
D b d i ??????– Dumb devices ??????
• Pervasive Computing– Monitoring and tracking almost anything
– The Internet is Free
• Continuous Services
• The Cloud + Pervasive Computing– Smart buildingsSmart buildings
– Sensor nets
– “Invisible computing” or “ubiquitous computing”
– Semantic Interoperability
– Ad hoc networking
Copyright 2012@ HBMG Inc.
4/1/2013
13
Advance Analytics as a strategic asset
“The future belongs to companies and people that turn data into products.”
Mike Loukides, O’Reilly
25
Advance Analytics as a strategic asset
“85% of eBay’s analytic workload is new and unknown. We are architected for the unknown.”
Oliver Ratzesberger, eBay
• Data exploration – data as the new oil The exploration for data, rather than the exploration of data
Uncovering pockets of untapped data
Processing the whole data set, without sampling
eBay’s Singularity platform combines transactional data with behavioral data, enabled identification of top sellers, driving increased revenue from those sellers
26
4/1/2013
14
Advance Analytics as a strategic asset
“Groupon will not be the first or last organization to compete and win on the power of data It’s happeningcompete and win on the power of data. It s happening everywhere.”
Reid Hoffman and James SlavetGreylock Partners
Data harnessing – data as renewable energy
H i t ll i d t t Harnessing naturally occurring data streams
Like harnessing raw energy to be converted into usable energy
Conversion of raw data into usable data
27
28
4/1/2013
15
BIG DATAREAL TIMEPREDICTIVEENABLED BYENABLED BYADVANCE ANALYTICS
As the world gets smarter, infrastructure demands will grow
Smart traffic
systems
Smart energy grids
Smart healthcare
Smart food
systems
Intelligent oil field
technologies Smart retail
Smart water management
Smart regions
Smart weather
Smart countries
Smart supply chains
Smart cities
4/1/2013
16
Data
Desktop
Knowledge
4/1/2013
17
Copyright 2012@ HBMG Inc.
The Nature of Communications Has Been Changing...
People toThings
People toPeople
Things toThings
People toThings
People toPeople
Things toThings
TOLL
TOLL
TOLL
TOLL
ThingsPeople ThingsThingsPeople Things
Copyright, 2011 © HBMG, Inc.
15.62 Gallons27.33 DollarsThank You!
15.62 Gallons27.33 DollarsThank You!
15.62 Gallons27.33 DollarsThank You!
4/1/2013
18
Mainframe/Terminal
Client Server
Ubiquitous
COSTS
Mainframe
Mini/Terminal
Server/PC
ASP/Appliance
USAGE
Distribution of Computing Power Virtualization Ubiquitous
Improved Access/Convenience
Improved Ease of Use
Large Organizations + Small Organizations + Departments + BYOD Entrepreneurs
Middle Class (PCs) + General Population
Digital Video Adapters
Satellite Radio Receivers
Digital CamerasPDAs
Wireless CamerasWireless TV Monitors
Digital Music Adapters
Networked Storage Centers
Game Consoles
Smart Displays
Smart Phones
Laptop PCs
Desktop PCs
Wireless Gaming Adapters
Movies-on-DemandReceivers
“Fourth Generation”Set-top Boxes
MP3 PlayersDigital Media Receivers Personal Video Recorders
Networked DVD Player Mobile Gaming Devices
802.11 Speakers
Copyright, 2012 © HBMG, Inc.
4/1/2013
19
The Global Grid
Copyright 2012@ HBMG Inc.
To Every Sensor is a Server
Phone -PDA
Processor
Data Storage
Communications
Rich variety of sensors
Smart Dust
Microphone
Microstorage
(Areal density 100x’s CD)
Copyright, 2011 © HBMG, Inc.
Embedded Biofluidics Chip
Robot
4/1/2013
20
3,500
4,000
• Mobile
Growth at the Edge of the Network
1,000
1,500
2,000
2,500
3,000
Pet
abyt
es/D
ay G
loba
lMobile
• Device to Device • Sensors • Entertainment• Smart Home• Distributed Industrial• Autos/Trucks• Smart Toys
ConvergedContent
0
500
2003 2004 2005 2006 2007 2008 2009 2010 2011
Year
2012
Traditional Computation
Copyright 2012@ HBMG Inc.
DOD ExampleDOD Example
4/1/2013
21
Counterinsurgency operations are complex increased emphasis on: Information and analysis at lowest levels
Shortened decision making time-scales
Operation Trends
Source: TTI Vanguard Conference - Psydex
g
Wider array of information sources
Continued growth in volume of data, especially informal information with limited structure must transform disparate info to knowledge
Processing power and storage capacity increasing faster than communications capacity must smartly position data and services within networks
Increased use of commercial cellular networks hybrid networks Increased use of commercial cellular networks hybrid networks that exploit and interoperate with commercial wireless comms is key
Enhancing coalition decision making depends on secure communications and information networks must address end-to-end problem of data-to-decision (coalition)
“Recce” P-38 “Recce” P-38
RF-101 Voodoo
Info in War Revolution
Technology—Information—Organization
B-17 Spotter Corps
0-2 Bird Dog
0-2 Bird DogB-52
75 Days75 Days
14 Days14 Days
<10 Minutes<10 Minutes
N b f W N b f S
RSI
RI
S ISR?
42
Evolution of Technology, Information, and Culture Enabled Move from Evolution of Technology, Information, and Culture Enabled Move from Segregation of Ops and Intel to Integration of Ops and Intel…Segregation of Ops and Intel to Integration of Ops and Intel…
Number of WeaponsRequired to Target
Number of SensorsRequired to Target
19431943 20092009
4/1/2013
22
National
StrategicStrategicStrategic
OperationalOperationalOperational
AIR
AIR
SP
AC
SP
AC
CY
BE
CY
BE
StrategicStrategic
OperationalOperational
StrategicStrategic
OperationalOperational
21st Century Challenges:Precision and Information Synergy
Tactical
OperationalOperational
TacticalTacticalTactical
DESERTSTORM
1991 & Prior
1999ALLIEDFORCE
2001ENDURING FREEDOM
2009AF/PAK & Iraq
TODAY
INFO AGEINFO AGEWARFAREWARFARE
TOMORROWTOMORROWRRC
EC
E
ER
ER
TacticalTactical
pp
TacticalTactical
Ops …+ Intel
I + S&R GLOBAL INTEGRATED
ISRSegregatedIntel & Ops
NTI
+ Intel INTEGRATED ISR
Intel & Ops
Kandahar Runway
Pod RecceReal‐TimeFusion
OPS
INTEL
Multi‐DomainFusion
More CollectorsMore CollectorsBetter SensorsBetter Sensors
Dimensions of ISR…“More of Everything”
More DataMore Data
•• More StorageMore Storage•• More CommsMore Comms•• More ToolsMore Tools•• More AnalystsMore Analysts•• More LinguistsMore Linguists
Better Intel
…All on an Operationally Responsive Timeline…All on an Operationally Responsive Timeline
4/1/2013
23
Sensor Data Volume
How do we handle all this data?How do we handle all this data?“Rebalancing Collection & PED may be Necessary”“Rebalancing Collection & PED may be Necessary”
Persistent Surveillance Data Rates
4/1/2013
24
General Norton A. Schwartz
Air Force’s chief of staff
“If i id f l h“If automation can provide a cue for our people that would make better use of their time, that would help us significantly,” NY Times.
Lieutenant General David Deptula
Former Deputy Chief of Staff for ISR, Headquarters, US Air p y f f ff f qForce
“We’re going to find ourselves in the not too distant future swimming in sensors and drowning in data”
Advance Analytics
• Advanced Analytics and Big Data are two of the most active areas of innovation in the Tech sector
• legacy infrastructures and government policies are increasingly at odds with the realities of the analytic landscape
• Certain forms of analysis is no longer possible within an encrypted environment. Rules that require data to be encrypted, both while in transit and at rest, also introduce performance penalties that make it difficult if not impossible to process large datasets in an acceptable timeframe
4/1/2013
25
Today's Cycle
Where is Real Time?
Advance Analytics
• The time to use the output is increasingly getting shorter – Real Time is becoming very common
Li it d il bl h d f i• Limited available human resources, and performance is often unreliable due to human fatigue and distraction. Therefore, automated real‐time sensor processing techniques are required to reliably detect and discriminate targets of interest
• Limited automated processing and tagging tools
• – Still NOT enough
4/1/2013
26
Advance Analytics
• The time to use the output is increasingly getting shorter – Real Time is becoming very common
• Limited available human resources, and performance is often punreliable due to human fatigue and distraction. Therefore, automated real‐time sensor processing techniques are required to reliably detect and discriminate targets of interest – Still NOT enough
• Need to move to thesensor/collector
• Needs to be embedded in the the sensor
Autonomous Systems
Agents coordinate and negotiate to achieve
common goals
Agents dynamically adaptto and learn abouttheir environment
Adaptive
Autonomous Mobile Interoperate
CooperativeSocial
PersonalityIntelligentAgents
InformationAgents
HBMG Inc. Copyright 2012
Agents are goal directedand act on theirown performing
tasks on your behalf
Agents moveto where they
are needed
Agents interoperatewith humans, other, legacy systems, and information sources
4/1/2013
27
Autonomic Networks
Self-configuring : Adapt automatically to the dynamically changing
Self-healing:Discover, diagnose, and react todynamically changing
environments of link and node failures.
Self-optimizing: Monitor and tune resources automatically during an attack to minimize its attack during and in the
and react todisruptions from catastrophes and attacks.
Self-protecting:Anticipate, detect, identify, and protect against attacks from anywhere (safety )
SelfSelf--HealingHealing
SelfSelf--ProtectingProtecting
SelfSelf--OptimizingOptimizing
SelfSelf--ConfiguringConfiguring
HBMG Inc. Copyright 2012
attack during and in the aftermath.
anywhere (safety.)
Numbers
• How many data in the world?
– 800 Terabytes, 2000
– 160 Exabytes, 2006
– 500 Exabytes(Internet), 2009
– 2.7 Zettabytes, 2012
– 35 Zettabytes by 2020
• How many data generated ONE day?
– 7 TB, Twitter
– 10 TB, Facebook
Big data: The next frontier for innovation, competition, and productivity
McKinsey Global Institute 2011
4/1/2013
28
1 illi1million transactions during this presentation
If You Liked ____, You’ll Love ___ !
4/1/2013
29
1billi1 billion clicks during this presentation
4/1/2013
30
Gartner Hype Cycle 2012
4/1/2013
31
2012 Business Intelligence, Analytics and Information Management Survey from InformationWeek Reports
A few insights from the report:
•58% of those surveyed are “very interested” in advanced analyticsy y y•Advanced analytics is the No. 1 leading-edge technology•Cloud analytics systems are hot because they are easier on the pocketbook; yet 63% of users have privacy concerns•Data pros just can’t get good data – data quality still ranks as the top barrier to adopting BI products throughout the company•25% of those surveyed are mobilizing their data analytics with dashboards and data visualizations40% f d t t li t t b th bi d t•40% of data pros are struggling to stay above the big data wave
Conclusion
Data is one the major factors driving infrastructure computing
The growing volume of data is a problem, but it is also an opportunity
Don’t worry about ‘big data,’ worry about your datay g , y y
Take a Total Data approach to data management
• Think pragmatically about data storage and analysis
• Attempt to capture and analyze any data that might be relevant, regardless of where it resides
‘Datastructure’ will become increasingly valuable, not only as a source of data but also as a source of intelligence
The rise of the ‘data cloud’ and the PaaS data layer will encourage a more flexible approach to data management and analytics
The companies that win will be those that think about data as a strategic asset and implement the technology to monetize it
4/1/2013
32
Conclusion
The World is moving to Real Time
Advanced Analytics is the Keyy y
Advanced Analytics Must be embedded in the collectors and sensors
• Think about where the data comes from
• Attempt to capture and analyze any data that might be relevant, regardless of where it resides
• Realize collaboration is the key in Advance Analytics just as it is in Business
If we don’t change ourIf we don t change our direction, we’ll end up exactly
where we are headed.
—Ancient Chinese Proverb
4/1/2013
33
In Parting: Be Paranoid
•“Sooner or later, something fundamental in your business world
•“Sooner or later, something fundamental in your business worldfundamental in your business world
will change.”
• Andrew S. Grove, Founder, Intel“Only the Paranoid Survive”
fundamental in your business world will change.”
• Andrew S. Grove, Founder, Intel“Only the Paranoid Survive”Only the Paranoid SurviveOnly the Paranoid Survive
Copyright @2008 HBMG Inc.