introduction to big data - สถาบันเพิ่มผลผลิต ......8/23/2017 6 11...
TRANSCRIPT
8/23/2017
1
Introduction toBig Data
Assoc. Prof. Dr. Thanachart Numnonda
Executive Director
IMC Institute
August 2017
1
2
Speaker
● Executive Director, IMC Institute
● Committee of the Council, Ubon
Ratchathani University
● Chairman, Siameast Solutions Public
Co.Ltd.
● Independent Director & President of
Audit Committee, Thanachart Bank
Public Co.Ltd.
● Independent Director, Vintcom
Technology Public Co.Ltd.
● Independent Director, Humanica Ltd.
2
8/23/2017
2
3
3
4
หุ่นยนต์เภสัชกร Wearable device
โทรศัพท์ฝั่งอยู่ในตัว
อินเตอร์เน็ต
เครื่องพิมพ์สามมิติพิมพ์อวัยวะคน
รถยนต์ไร้คนขับSharing car
4
8/23/2017
3
5
5
6
Every activities create data
6
8/23/2017
4
7
7
8
8
8/23/2017
5
9
9
10
10
8/23/2017
7
13
In an era of the Internet of Things, companies that have the right IT architecture and infrastructure, and the right talent, capable of both handling fast-moving technologies and finding meaning in big data, will be able to leapfrog their competitors.
Harvard Business Review
13
14
14
8/23/2017
8
15
Big Data
Big Data Analytics
Data Science
Machine Learning
Artificial Intelligence
Deep Learning
15
16
Big Data
Source: http://www.datasciencecentral.com/
8/23/2017
9
17Source: IBM
18Source: IBM
8/23/2017
10
19Source: IBM
20Source: IBM
8/23/2017
11
21Source: William EL KAIM, Enterprise Architecture and Technology Innovation
Big Data : Why Now?
21
22
22
8/23/2017
12
23Source: Big Data Analytics: The Revolution Has Just Begun, SAS Software
24Source: Bernard Marr
8/23/2017
13
25
25
26
26
8/23/2017
14
27
We are forecasting the future based on the past
27
28
28
8/23/2017
15
29Source Big Data Analytics with Hadoop: Phillippe Julio
Use Cases
30
WALMART: Retail Industry
Problem Solving
● Realtime analytics:
product recommendation
● Right place, right time,
right customer
● Monitors public social
media conversations,and
attempts to predict what
products people will buy
- Largest retailer in the world- 20,000 stores in 28 countries.- Has Big Data and analytics department since 2004- The world’s largest private data cloud- Process 2.5 PB every hour
Source: Big Data in Practice, Bernard Marr, 2016
30
8/23/2017
16
31
WALMART: Retail Industry
Technology
● 40 petabytes of data
● Hadoop (since 2011)
● Spark
● Cassandra
● R
● SAS
Data
● Data Café uses database
consisting of 200 billion
rows of transactional data
● 200 other sources,
including meteorological
data, economic data,
telecoms data, social
media data, gas prices
Source: Big Data in Practice, Bernard Marr, 2016
31
32
WALMART: Retail Industry
Results
● Data Café system has led to a reduction in the time it takes
from a problem being spotted in the numbers to a solution
being proposed from an average of two to three weeks down
to around 20 minutes.
Source: Big Data in Practice, Bernard Marr, 2016
32
8/23/2017
17
33
33
34
Netflix: Entertainment
Problem Solving
● To understand customer
viewing habits
● Improve in the number of
hours customers spending
● They launched the Netflix
Prize
- Streaming movie and TV service - 65 million members in over 50 countries- one-third of peak-time Internet traffic in the US
Source: Big Data in Practice, Bernard Marr, 2016
34
8/23/2017
18
35
Netflix: Entertainment
Technology
● 40 petabytes of data
● Amazon Web Services
● Hadoop, Hive and Pig
● Originally used Oracle
databases, but they
switched to NoSQL and
Cassandra
Data
● Customer ID, movie ID,
rating and the date the
movie was watched
● Streaming data
Source: Big Data in Practice, Bernard Marr, 2016
35
36
36
8/23/2017
19
37
Netflix: Entertainment
Results
● They added 4.9 million new subscribers in Q1 2015,
compared to four million in the same period in 2014.
● Q1 2015 alone, Netflix members streamed 10 billion hours of
content.
Source: Big Data in Practice, Bernard Marr, 2016
37
38
Uber: Transportation
Problem Solving
● Big Data principle of
crowdsourcing.
● Store and monitor data on
every journey to determine
demand, allocate resources
and set fares.
● Big Data-informed pricing,
which call “surge pricing”
- A smartphone app-based taxi booking service.- Now valued at $41 billion. - Firmly in Big Data, and leveraging this data in a more effective way than traditional taxi firms.
Source: Big Data in Practice, Bernard Marr, 2016
38
8/23/2017
20
39
Uber: Transportation
Technology
● Hadoop data lake.
● Apache Spark
Data
● mixture of internal and
external data.
● GPS, traffic data
● public transport routes
Source: Big Data in Practice, Bernard Marr, 2016
39
40
Uber: Transportation
Results
● This case is less about short-term results and more about
long-term development of a data-driven business model. But
it’s fair to say that without their clever use of data the
company wouldn’t have grown into the phenomenon they
are.
Source: Big Data in Practice, Bernard Marr, 2016
40
8/23/2017
21
41
41
42
Amazon
Problem Solving
● “recommendation engine”
technology is based on
collaborative filtering.
● “360-degree view” of you
as an individual customer
● monitor, track and secure
its 1.5 billion items in its
retail store
- one of the world’s largest retailers of physical goods, virtual goods such as ebooks and streaming video and more recently Web services.
Source: Big Data in Practice, Bernard Marr, 2016
42
8/23/2017
22
43
Amazon
Technology
● 187 million unique
monthly website visitor.
● Hewlett-Packard servers
running Oracle on Linux
● 5 TB of data
Data
● Data from users as they
browse the site.
● Location data and
information about other
apps use on your phone.
● External datasets such as
census information
● Streaming data
Source: Big Data in Practice, Bernard Marr, 2016
43
44
Amazon
Results
● Amazon have grown to become the largest online retailer in
the US based on their customer-focused approach to
recommendation technology. Last year, they took in nearly
$90 billion from worldwide sales.
Source: Big Data in Practice, Bernard Marr, 2016
44
8/23/2017
24
47
Technology
Analytics
Data Sources
48
Data, Data Everywhere
• Structure data
• Semi-structure data
• Unstructure data
• Internal data
• External data
• Activity data
• Conversation data
• Photo data
• Sensor data
• IoT data
8/23/2017
25
49
DataCollection
DataStorage
DataAnalysis/Processing
Datavisualisation
Technology
50
Big Data Technology
8/23/2017
26
51
Source: Data Science and Critical Thinking, A, Croll
The old way: Ask, then collect
The new way: Collect, then ask
51
52
Big Data
Data
WarehouseBI
Data LakeData
Science
52
8/23/2017
27
53
54Image: rodneyrohrmann.blogspot.com
8/23/2017
28
55
Data Warehouse
Source: dinesql.blogspot.com
56
Differences between Data Lake and
Data Warehouse
Source: martinfowler.com/bliki/DataLake.html
8/23/2017
29
57
Technology
Analytics
Data Sources
58
Data, Data Everywhere
• Structure data
• Semi-structure data
• Unstructure data
• Internal data
• External data
• Activity data
• Conversation data
• Photo data
• Sensor data
• IoT data
8/23/2017
30
59
DataCollection
DataStorage
DataAnalysis/Processing
Datavisualisation
Technology
60
Big Data Technology
8/23/2017
31
61
Source: Data Science and Critical Thinking, A, Croll
The old way: Ask, then collect
The new way: Collect, then ask
61
62
Big Data
Data
WarehouseBI
Data LakeData
Science
62
8/23/2017
32
63
64Image: rodneyrohrmann.blogspot.com
8/23/2017
33
65
Data Warehouse
Source: dinesql.blogspot.com
66
Differences between Data Lake and
Data Warehouse
Source: martinfowler.com/bliki/DataLake.html
8/23/2017
34
67
Don't Think Big Data Technology, Think Business Transformation
67
68
Don't Think Business Intelligence, Think Data Science
68
8/23/2017
35
69
Don't Think Data Warehouse, Think Data Lake
69
70
Don't Think “What Happened,” Think “What Will Happen”
70
8/23/2017
36
71
How to Use Big Data?
• Using data to make better business decisions
• Using data to improve your business operations
• Transforming your business model: data as a
business asset
71
72
Big Data Business Model Maturity Index
Source: Big Data MBA, Bernard Marr
72
8/23/2017
37
73
www.facebook.com/imcinstitute
73
74
74
8/23/2017
38
75
Thank you
www.facebook.com/imcinstitute
www.slideshare.net/imcinstitute
www.thanachart.org
75