Download - Michael newberry

Transcript
Page 1: Michael newberry

Extracting Value from Big Data in the Cloud -

Michael Newberry

Page 2: Michael newberry

Big data in a Hybrid-Cloud worldDr Michael Newberry

Windows Azure Lead, Microsoft [email protected]

Page 3: Michael newberry
Page 4: Michael newberry

Doggerland: Simon Fitch, Vince Gaffney and Ken ThomsonImage Source: drowned-landscapes.tumblr.comRoyal Society's Summer Science Blog (http://summer-science.tumblr.com/)

Page 5: Michael newberry

Big Data.

Page 6: Michael newberry

Big Data.

VOLUME (Size)

VARIETY (Structure)

VELOCITY (Speed)

Page 7: Michael newberry

Getting useful insightsfrom awkward data setsusing the most appropriate computing platform at each stage.

Dr Michael NewberryWindows Azure LeadMicrosoft UK

Page 8: Michael newberry

Big data in a Hybrid-Cloud worldDr Michael Newberry

Windows Azure Lead, Microsoft [email protected]

Page 9: Michael newberry

Machine Learning & Bayes theorem

𝑝 ( h𝑀𝑖𝑐 π‘Žπ‘’π‘™π‘π‘’π‘¦π‘–π‘›π‘”π‘ π‘œπ‘π‘˜π‘  𝑖𝑓h𝑒hπ‘Žπ‘  𝑗𝑒𝑠𝑑 hπ‘π‘œπ‘’π‘” 𝑑 h𝑠 π‘œπ‘’π‘ )π‘‘π‘’π‘π‘’π‘›π‘‘π‘ π‘œπ‘›

𝑝 ( h𝑀𝑖𝑐 π‘Žπ‘’π‘™π‘π‘’π‘¦π‘–π‘›π‘”π‘ π‘œπ‘π‘˜π‘  )𝑝 ( h𝑀𝑖𝑐 π‘Žπ‘’π‘™π‘π‘’π‘¦π‘–π‘›π‘” h𝑠 π‘œπ‘’π‘  )

𝑝 ( 𝐴∨𝐡 )=𝑝 (𝐡∨𝐴 ) 𝑝 ( 𝐴 )𝑝 (𝐡 )

Page 10: Michael newberry

….Amazon (AMZN) calls this homegrown math "item-to-item collaborative filtering," and it's used this algorithm to heavily customize the browsing experience for returning customers…. Judging by Amazon's success, the recommendation system works. The company reported a 29% sales increase to $12.83 billion during its second fiscal quarter, up from $9.9 billion during the same time last year. A lot of that growth arguably has to do with the way Amazon has integrated recommendations into nearly every part of the purchasing process from product discovery to checkout.

http://tech.fortune.cnn.com/2012/07/30/amazon-5/

Page 11: Michael newberry

β€œIn theory there is no difference between theory and practice; in practice, there is”.

Yogi Berra, cited in Nassim Taleb, Antifragile.

Page 12: Michael newberry

Big data techniques

NoSQL (ala MongoDB) Map-Reduce (e.g. Hadoop)

Page 13: Michael newberry

Embedded devices

Connected Devices

On Premise

Off Premise

Business Intelligence

Customers Employees, Partners

The Power of an Intelligent System

Page 14: Michael newberry
Page 15: Michael newberry

Modern Platform for the World’s Apps

Cloud OS

transforms the datacenterenables modern appsunlocks insights on any dataempowers people-centric IT

Page 16: Michael newberry

Cloud OS

flexible developmentunified dev-ops & managementcomplete data platformcommon identityintegrated virtualization

MICROSOFT

SERVICE PROVIDERON-PREMISES

1CONSISTENTPLATFORM

What Makes the Cloud OS Unique

Page 17: Michael newberry

RelationalNon-Relational Streaming

MANAGE ANY DATA, ANY SIZE, ANYWHERE

010101010101010101101010101010101001010101010101101010101010

Unified Monitoring, Management & Security

Data Movement

Page 18: Michael newberry

POLYBASE: COMBINING RELATIONAL AND NON-RELATIONAL DATAThe future of query processing

select... results set

Hadoop Data Warehouse

PolyBase

Single query for relational & Hadoop data

Process data in place

Future expansion to other data sources

Seamless: regular T-SQL command

Page 19: Michael newberry

19

Page 20: Michael newberry

20

Page 21: Michael newberry

Avoiding Lock-InWindows Virtual machines can move freely between all 3 clouds.

Windows Azure

Customer Data Center

Other Service ProvidersWindows

Virtual Machine

Page 22: Michael newberry

LocationOn-Premises On-Premises or

Service ProviderMicrosoft Cloud orService Provider

Rationale for Usage

Compliance

Scalability

Economies of Scale

Rapid Development

Complex, Legacy Applications

Compliance

Economics

TraditionalNON-VIRTUALIZED

AppliancePRIVATE

CloudPUBLIC

(Outside Firewall)

DATA PLATFORM DELIVERY MODELS

(Inside Firewall)

Page 23: Michael newberry

BALANCING ON PREMISE & CLOUDSnowline graph

Page 24: Michael newberry

A

Page 25: Michael newberry

Takeaways

1. β€œbig data” can do some amazing stuff.2. Don’t think β€œbig data” as much as β€œdata needing non-

relational approaches”3. If your big data insights are probabilistic, which they often are,

have a plan to deal with variance. 4. Pick the most appropriate platform: Think β€œand” not β€œor”:

- Balance public cloud AND on-premise,- Combine β€œbig data” with RDBMS.

Page 26: Michael newberry

Q+A


Top Related