there is more to big data than data
DESCRIPTION
TRANSCRIPT
There is more to Big Data than data.
September 30th – October 4th
Big Data Survey Results
Accelerating Innovation & Time to Value
695,000 status updates
98,000+ tweets
698,445 Google searches
1,820TB of data created
11million instant messages
168 million+ emails sent
YouTube
Viber
Qzone
Amazon Web Services
GoGrid
Rackspace
LimeLight
Jive Software
salesforce.com
Xactly
Paint.NET
Business
EducationEntertainment
Games
Lifestyle
Music
Navigation
News
Photo & Video
Productivity
Reference
Social Networking
Sport
Travel
Utilities
Workbrain
SuccessFactors
Taleo
Workday
Finance
box.net
TripIt
Zynga
Zynga
Baidu
Yammer
Atlassian
Atlassian
MobilieIronSmugMug
SmugMug
Atlassian
Amazon
AmazoniHandy
PingMe
PingMe
Associatedcontent
Flickr
Snapfish
Answers.com
Tumblr.
Urban
Scribd.Pandora
MobileFrame.com
Mixi
CYworld
Renren
Yandex
Yandex
Heroku
RightScale
New Relic
AppFog
BromiumSplunk
CloudSigma
cloudability
kaggle
nebula
Parse
ScaleXtreme
SolidFire
Zillabyte
dotCloud
BeyondCore
Mozy
Fring Toggl
MailChimp
Hootsuite
Foursquare
buzzd
Dragon Diction
SuperCam
UPS Mobile
Fed Ex Mobile
Scanner Pro
DocuSign
HP ePrint
iSchedule
Khan Academy
BrainPOP
myHomework
Cookie Doodle
Ah! Fasion Girl
PaperHost
SLI Systems
NetSuite
OpSource
Joyent
Hosting.com
Tata Communications
Datapipe
PPM
Alterian
Hyland
NetDocuments
NetReach
OpenText
Xerox
Microsoft
IntraLinks
Qvidian
Sage
SugarCRM
Volusion
Zoho
Adobe
Avid
Corel
Microsoft
Serif
Yahoo
CyberShift
Saba
Softscape
Sonar6
Ariba
Yahoo!
Quadrem
Elemica
Kinaxis
CCC
DCC
SCMADP VirtualEdge
Cornerstone onDemand
CyberShift
KenexaSaba
Softscape
Sonar6
Workscape
Exact Online
FinancialForce.com
IntacctNetSuite
Plex Systems
Quickbooks
eBay
MRM
Claim Processing
Payroll
Sales tracking & Marketing
CommissionsDatabase
ERP
CRM
SCM
HCM
HCM
PLM
HP
EMC
Cost Management
Order Entry
Product Configurator
Bills of MaterialEngineering
Inventory
Manufacturing Projects
Quality Control
SAP
Cash Management
Accounts ReceivableFixed AssetsCosting
Billing
Time and Expense
Activity ManagementTraining
Time & Attendance
Rostering
Service
Data Warehousing
The Internet Gigabytes!
Client/Server Megabytes!
Every 60 seconds
IBM
Unisys
Burroughs
Hitachi
NECBull
Fijitsu
Mainframe Kilobytes
Mobile, Social, Big Data & The Cloud Zettabytes!
217 new mobile web users
Yottabytes
Business challenge – Opportunities lost
% of the Digital Universe that actually is being tagged and
analyzed
Competitive Advantage in the Digital Universe in 2012 Massive amounts of useful data are getting lost
23% 3%% of data that would be potentially useful
IF tagged and analyzed
% actually being tagged for Big Data Value (will grow to 33% by 2020)
0.5%¹Source: IDC The Digital Universe in 2020, December 2012
Technology challenge Legacy techniques have all fallen short.
Stale technologies Talent shortage
86%!of corporations cannot deliver the right information, at the right time to support enterprise outcomes all of the time³
³Source: Coleman Parkes Survey Nov 2012
IT frustration Lack of insight
Lower cost and demonstrate value to the business
Key Market Drivers
Massive growth in volume and variety of data
High expectations for up-to-date data
Data access for Business Analysts
How do I know the data is current?
How can I make sense of all this data?
How do I align this data for business insights?
Big Data challenges that are often overlooked
1. Did you just call my baby ugly?
2. That’s not what we used before!
Credit Card Risk Analysis PoC Results Summary
Measure Data Mart Vertica PoC Factor
Rows 365 MM 857 MM 2.56Table Space 338 Gbyte 983 Gbyte 2.91BI 74 Seconds 1.62 Seconds 45.68Transaction 77 Seconds 0.48 Seconds 160.42Scan 65 Minuets 52.0 Seconds 75.00Extreme Not Attempted 5.68 seconds -------Virtual Mart Not Attempted 52.4 seconds -------
Result Summary• PoC contained almost three times (3X) the data volume of Data Mart • PoC queries executed 93 times faster on average than Data Mart • PoC was implemented on a single node with commodity (HP) hardware
Big Data challenges that are often overlooked
1. Did you just call my baby ugly?
2. That’s not what we used before!
3. It’s just commodity hardware, right?
HP server speeds into the Guinness World Record books
The ProLiant DL 980 recently earned a Guinness World Record by loading and analyzing 34.3 terabytes of structured and unstructured data per hour in a system configuration that also uses the HP 3PAR StoreServ storage array technology.
Just how fast is 34.3 terabytes per hour? So fast that it could load the entire Library of Congress in under 35 minutes. That’s more than 155 million items, including more than 23 million books, 68 million manuscripts, and much, much more.So what can our customers do with the DL980? • California Department of Health Care Services was able to replace 400 of a
competitor’s servers with just 4 DL980s • Netherlands customer reduced their data center footprint by 16 square meters and its
power consumption by 87.5% • Bank Sinopac was able to cut bulk transactions execution time from 8 hours to 4
hours.
Project Kraken
▪ Meeting SAP needs for a technology platform to handle large amounts of in-memory computing resources%• Leadership platform%• Biggest ever SAP HANA machine%• Development platform for next
generation SAP HANA (e.g., DragonHawk)%
• Mission-critical environment%%▪ 2013 at SAP press event
Demonstrating SAP Business Suite on SAP HANA
Watch: SAP Business Suite on HANA
“…we co-labored with HP to demonstrate an x86-based 160 core 8 TB machine, nick-named Kraken running 200 B scans per second…similar to the HP system we put together 20 years ago, just been updated with latest technology…” %– Dr. Vishal Sikka, CTO and executive board member, SAP on 10 Jan
HP AppSystem for Apache Hadoop – Cloudera
%• Easier to deploy & scale with unique HP Cluster
Manager%%
• Easier to manage with real time performance visualization%%• Faster data analysis with ProLiant Gen8 platform%%
• Faster loading, sorting, and analysis - 10TB at 120GB/min
3.8x!Times Faster
800!Nodes in
minutes not months
1!Push-button deployment
5x!Memory
Project Mercury: eBay runs Hadoop on HP
That’s a lot of data!! !
• Over 300 million items for sale
• More than 100 million active users
• Over $2,000 in transactions/second !
%The Next Generation Data Center! !
• Two 24 Petabyte Hadoop clusters
• Largest data warehouse expansion in eBay’s history
• Data capacity grew by 500% over six months
%
“We have gone all in on Big Data, because data is absolutely king” - Dean Nelson, Senior director of global foundation services, eBay
Big Data challenges that are often overlooked
1. Did you just call my baby ugly?
2. That’s not what we used before!
3. It’s just commodity hardware, right?
4. Is your solution sustainable?
HP Moonshot SystemThe world’s first software defined server
77% Less cost
80% Less space
97% Less complexity
89% Less energy
HP Moonshot 1500 Chassis Supports shared components including power, cooling, and management and fabric %Software defined servers 45 individually serviceable hot-plug cartridges
Source: HP internal analysis
One node
Unmatched storage density with 60 LFF drives
in a single node per chassis
SL4540 & SL4545%(1X60)
SL4540 & SL4545 %(3X15)
SL4540 & SL4545%(2X25)
Two node Three node
Ideal combination storage density and compute with 25 LFF drives per node in a dual
node chassis
Optimal storage and compute with 15 LFF drives per node
in a triple node chassis
Purpose built storage density and efficiency for scale out storage
HP ProLiant SL4500 HyperStorage Series
Object Storage & DB
Email & Hadoop MongoDB & Hadoop
Fastest Time to Value; Purpose Built for Big Data Scale and Performance
HP is leading in Big Data innovations
AppSystem for Apache Hadoop
AppSystem for Vertica
AppSystem for SAP HANA
AppSystem for Microsoft PDW
AppSystem for Autonomy
HP solutions deliver more choice to meet specific workloads , data volumes and variety versus our competitors’ “one-size-fits-all”
approach
HAVEn – Big Data Platform
HAVEn
Social media IT/OT ImagesAudioVideoTransactional
dataMobile Search engineEmail Texts
Catalog massive volumes of distributed data
Hadoop/HDFS
Process and index all information
Autonomy IDOL
Analyze at extreme scale in real-time
Vertica
Collect & unify machine data
Enterprise Security
Powering HP Software + your apps
nApps
Documents
Questions