emc big data: redefiniendo aplicaciones · pdf file1 zb = 1b tbs 44 zettabytes equivale a 50...
TRANSCRIPT
1 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
EMC Big Data: Redefiniendo Aplicaciones
2 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
“In the beginning was the command line” Neal Stephenson
3 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Enterprise Internet
Big Data?
4 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Evolución o Revolución del Big Data?
5 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
¿Cuántos Datos?
7.6B people
200B things
44 Zettabytes 1 ZB = 1B TBs
44 zettabytes equivale a
50 veces los granos de arena de todas las playas en la
Tierra.
6 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Big Data Importa!
Entendiendo el Comportamiento del cliente
Gestión de Riesgo
Optimizar Operaciones
Facilitador Innovación
7 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Big Data: entendiendo al cliente
9 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Ejemplos Cliente EMC Entendiendo el comportamiento del cliente
Easynet enables retailer to increase revenue per customer by 5% through improved customer loyalty program
Knotice enables retailers to increase conversion rates by 700% on Black Friday through improved customer ad targeting
Havas Digital enabled a Travel company to increase sales by 27% and ROI by 300% though better campaign optimization
10 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
¿Por qué ahora?
“ Through 2015, organizations integrating high-value, diverse, new information types and sources into a coherent information management infrastructure will outperform their industry peers financially by more than 20%.”
“ We built what looks like a software company and we're moving from silos to a single platform.”
“ The shift to digital requires a complete overhaul of banks technology…it is a matter of survival...we now have a state of the art platform.”
11 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Personas
Organización
Tecnologia
Claves: P.O.T.
• Casos de Negocio poco claros
• Escasez de recursos
• Falta de Experiencia
• Rigidez desarrollo de App
• Distribución compleja de App
• Data silos
• Costes de gestión
Situación Actual Modelo Orientado a Datos
• Casos de uso Optimizados
• Equipos preparados y
experimentados
• Metodologia AGILE
• PaaS
• Data Lake
• Gestión de datos simplificada
12 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Clave: Personas
Situación actual EMC Solutions
• EMC Big Data Curriculum
• Pivotal Data Labs
Personas
Organización
Tecnologia
• Casos de Negocio poco claros
• Escasez de recursos
• Falta de Experiencia
• Rigidez desarrollo de App
• Distribución compleja de App
• Data silos
• Costes de gestión
13 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Recursos…Dentro y fuera de las organizaciones
http://www.mastersindatascience.org
14 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
EMC Big Data Curriculum Requisitos y conocimientos para abordar proyectos
90 min
1 day
5 days
Data Science and Big Data Analytics
Data Science and Big Data Analytics for Business Transformation
Introducing Data Science and Big Data Analytics for Business Transformation
15 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Pivotal Data Labs Metodologia y experiencia de cientificos de datos para nuevos proyectos
Discovery Insights Results
1-12 week engagement
16 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Resultados Improved customer retention through faster identification of
at-risk customers
Easily scaled from 6 to 11 terabytes of data
Objetivos Better understand and serve customers utilizing high
volume, new data sets
Cost-effective means of accommodating database growth and complex data analysis
Soluciones EMC Data Computing Appliance (DCA)
Servicios Pivotal Data Labs
Ejemplo: De la idea a la solución
17 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Claves: Organización
Organización
Procesos
Tecnologia
Situación Actual EMC Soluciones
• EMC Big Data Vision
Workshop
• Pivotal Labs
• Casos de Negocio poco
claros
• Escasez de recursos
• Falta de Experiencia
• Rigidez desarrollo de App
• Distribución compleja de App
• Data silos
• Costes de gestión
18 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Document
•
•
Prioritize
•
•
Ideation
•
Analysis
•
•
Research
•
Recommendation
Improve predictive models
Ease of data Acquisition
Cost of Acquisition
Data Management /
Preparation
Machine sensor logs / error codes
Digitalized Work Orders
Machine vibration data
Manufacturer Performance History
Omega machine maintenance data
Other providers maintenance data
Location-based data
What If… Deliver Real-time, Personal Offers Integrating Customers’ Shopping Propensities And Current Location?
XXXX.XX
XXXX.XX
XXXX.XX
Shop Hot Offer! >
What are the usage patterns of my most “valuable” card members?
What are the usage patterns that indicate someone may churn?
How do I gain insights into cardmember’s interests, passions, affiliations and associations?
How do I leverage personalized offers to increase cardmember engagement and usage?
What additional insights would my Merchants value?
Hi Lo
Hi
I m plem entat ion Feasibility
Bu
sin
es
s V
alu
e
Churn: Leverage customer usage data to improve Churn Predictive Model Effectiveness
Product Perform ance: Change network
bandwidth based upon customer’s usage patterns
Netw ork Opt im izat ion: Optimize Network
investments using customers apps usage patterns
Standardizat ion: Standardize tools, processes, analytic models and hiring profiles across teams
Recom m endat ions: Create product recommendations based upon usage behaviors
Monet izat ion: Leverage/package customer usage data to drive new monetization opportunities
A
B
C
D
E
F
F
E
C
A
B
D
Monet ize Custom er Usage Behaviors
Sesión de Trabajo de 1 dia (2 semanas de trabajo preliminar)
EMC Big Data Vision Workshop Proceso colaborativo para definir casos de uso BIG DATA
19 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Pivotal Labs
• Agile development practices enable quick response to market changes
• Pivotal Tracker allows for full control over projects
• Collaborative, paired programming approach delivers better product in less time
Metododologia AGILE que permite reducir el ciclo de vida
QA
Release
Feedback Define/Pr
ioritize
Code
Build
20 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Ejemplo: Desarrollo App con Agile
Results Helped launch service & define development practices
GNIP able to lead the public social data ecosystem WW
Captured the business of 90% of the Fortune 500
Objectives Create a SaaS solution with rich features around ever-expanding universe of
social media data
Provide a consistent and reliable architecture to gain insight from real-time data sourced from Twitter, Facebook, Tumblr, WordPress, Instagram and more
Solutions Pivotal Labs (Agile development practices)
Pivotal Tracker (Project management and collaboration)
21 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Clave: Tecnologia
Situación Actual EMC Solutions
• EVP Data Lake
• Pivotal CF
Procesos
Organización
Tecnologia
• Casos de Negocio poco claros
• Escasez de recursos
• Falta de Experiencia
• Rigidez desarrollo de App
• Distribución compleja de
App
• Data silos
• Costes de gestión
22 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Arquitecturas de App Analíticas actuales Verticales y caras
Enterprise Apps
Reporting
Data Marts
Prioritized Operational Processes
Data Warehouse
Data Sources
Non-Prioritized Data Provisioning
Cloud Services
23 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Arquitectura Abierta: Construyendo Data Lake Unificar los servicios centralizados de Almacenamiento de datos, proceso y aplicaciones
Ingest Store Analyze Surface Act
Capture data from a wide range
of sources, traditional and
new.
Store everything in one
environment for cross data-set
analysis.
Use advanced algorithms to discover new,
predictive patterns.
Share insight with business domain
experts.
Build data-driven applications that meet business
needs.
24 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Data Lake????
https://infocus.emc.com/william_schmarzo/data-lake-data-reservoir-data-dumpblah-blah-blah/
25 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
EVP Data Lake Plataforma abierta para aplicaciones
D A T A L A K E
VOLUME NFS
S3 SWIFT ATMOS
VARIETY NFS
SMB
VELOCITY NoSQL
• Modular architecture enables use of some or all components
DATA
DSSD
ISILON
ViPR ECS APPLIANCE
VNX
OTHER
ViPR
ANALYTICS
GEMFIRE XD
HAWQ
PIVOTAL HD
DCA
HDFS
HDFS
HDFS
HDFS
APPLICATIONS
INTERACTIVE
BATCH
VMWARE
CLOUDFOUNDRY
REALTIME
IN-MEMORY SQL
SQL
MR
HDFS
NoSQL
• Existing data available for analytics through HDFS
• Multi-protocol support enables legacy applications use
• Enables different data processing needs
30 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Distribución Continua de App: Pivotal CF
• Developers can focus on development, not infrastructure
• Divide AppDev from Operations
• Eliminate provisioning and deployment bottlenecks
Delivers A Turnkey PaaS Experience With Leading App And Data Services
Public Private
Hybrid
31 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Ejemplo: Solución de Data Lake
Results Rapidly launched new service
Delivered high performance & scalability with simple administration & management
Objectives Rapidly launch new market intelligence service for fashion
retailers
Support large and growing volumes of Big Data
Solutions Pivotal Greenplum Database
Pivotal HD
EMC Isilon
Pivotal Data Labs
32 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
33 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
Tecnología: EMC ofrece las ultimas y mejores soluciones tecnológicas para
simplificar las arquitecturas de datos y evolucionar al Data Lake
Organización: EMC ofrece metodología contrastada para implementar procesos de
negocio mediante Big Data
Personas: EMC ofrece personal experimentado en Big Data/Data Science para enseñar a los equipos a desarrollar
proyectos Big data
Por qué EMC?
34 © Copyright 2014 EMC Corporation. All rights reserved. © Copyright 2014 EMC Corporation. All rights reserved.
“At the end, human information” “In the beginning was the command line”