Cloud DataDave CampbellTechnical Fellow, Microsoft

Agenda

Starting with the basics

Build and deploy

Community led data topics

Starting with the basics

Storage options

Blob StorageSQL Azure DatabaseTable Storage

Historical storage options

Available in most operating systems:Files (BLOBs)Indexed Record StorageRDBMSQueues

Azure Storage Options

Everything is Available at the Portal

Storage options

Blob StorageSQL Azure DatabaseTable Storage

What is the BLOB service?

Contained within Storage AccountNamespaceRESTful InterfaceGET/PUT BLOBs

Page & Block BLOBs

Storage options

Blob StorageSQL Azure DatabaseTable Storage

What is the Table Service?

SimpleREST accessibleHighly scalableCost effectiveNon-relationalHierarchical

Tables store entitiesEntity schema can vary in the same table

Storage Account: MovieData

Star WarsMatrixFan Boys

Table Name: Movies

Mike CollierMr. AndersonBill Gates

Table Name: Actors

Entity

Table Account

Using Azure Tables

demo

Storage options

SQL Azure DatabaseBlob Storage Table Storage

What is SQL Azure Database?

SQL Azure Database isSQL Server database technology delivered as a service on the Windows Azure PlatformIdeal for both simple and complex applicationsEnterprise-ready with automatic support for HADesigned to scale out elastically with demand

Provision your server

Server definedService head that contains databases

Connect via automatically generated FQDN (xxx.database.windows.net)

Initially contains only a master database

Provision servers interactively

Log on to Windows Azure Management Portal

Create a SQL Azure server

Specify admin login credentials

Add firewall rules and enable service access

Automate server provisioning

Use Windows Azure Platform PowerShell cmdlets (or use REST API directly)

wappowershell.codeplex.com

SQL Azure pay as you go pricing

Standard pay-as-you-go pricing

$9.99 per 1 GB of database per month

Business Edition (up to 150 GB)

$99.99 per 10 GB of database per month (Maximum charge of $499.95 per database)*

Database

Web Edition (up to 5 GB)

Build and Deploy your database

Build your databaseUse familiar technologies

Supports Transact-SQLSupports popular languages

.NET Framework (C#, Visual Basic, F#) via ADO.NETC / C++ via ODBCJava via Microsoft JDBC providerPHP via Microsoft PHP provider

Supports popular frameworksOData (REST data access)Entity FrameworkWCF Data ServicesNHibernate

Supports popular toolsSQL Server Management Studio (2008 R2 and later)SQL Server command-line utilities (SQLCMD, BCP)CA Erwin® Data ModelerEmbarcadero Technologies DBArtisan®

Differences in comparison to SQL Server Focus on logical vs. physical administrationDatabase and log files automatically placedThree high-availability replicas maintained for every databaseDatabases are fully containedTables require a clustered indexMaximum database size is 150 Gb

Unsupported SQL Server featuresBACKUP / RESTOREUSE command, linked servers, distributed transactions, distributed views, distributed queries, four-part namesService BrokerCommon Language Runtime (CLR)SQL Agent

Build your databaseThin client database development

Introducing the SQL Azure management portal

Web designers for tables, views, stored procs

Interactive query editing and execution

Rich client database development

Introducing SQL Server Data Tools (SSDT)

Visual Studio IDE for database development

Includes modern designers and projects with declarative, model-driven development

Develop and test in both connected and disconnected states

Platform targeting for both SQL Server (2005 and above) and SQL Azure

Get it free with Web PI, with SQL Server 2012 and with Visual Studio 11

Creating a SQL Azure DB

demo

Moving data to your database with DAC Data-tier Application Framework

(DAC Fx)Alternative to traditional script based approach

Dramatically simplifies deployment, migration and versioning of databases

Provides a single unit of deployment for schema (dacpac) or for schema + data (bacpac)

Supports automatic versioning of database schemas

Supports platform targeting for both SQL Server (2005 and above) and SQL Azure

Build from scratch or extract from existing db

How to get the latest DAC FxWith SQL Server Data Tools

With SQL Server 2012 Management Studio

With SQL Azure Import/Export Service

Via sqldacexamples.codeplex.com

Deploy your databaseInteractive approach for dacpac v1 and v2

Manage your server with SQL Azure Management Portal

Deploy dacpac

Load data (scripts, bulk copy, SSIS)

Also supported in SQL Server 2012 Management Studio

Interactive approach for bacpac v2Upload bacpac to blob storage

Log into Windows Azure Management Portal

Import bacpac, table data automatically loaded

Also supports export to blob storage

Upgrading a dacpac or bacpacNot supported in portals yet

Use SQL Server 2012 Management Studio

Scale out your data

SQL Azure Federations provides

Integrated database sharding that can scale to hundreds of nodes

Multi-tenancy via flexible repartitioning

Online split operations to minimize downtime

Automatic data discovery regardless of changes in how data is partitioned

Special considerationsA logical database can contain multiple federations

Distribution scheme supports int, bigint, guid, and varbinary types

Filtering routes connection to appropriate shard regardless of changes in partitions

Merge, fan-out queries and automatic distribution of schema changes not supported in initial release

Multi-Tenant Db

Tenant 1

Tenant 2

Tenant 3 Db Tenant 4

Tenant 4 Db 1

Tenant 4 Db 2

Federation Root

SQL Azure Federations

demo

Community led data topics

What About noSQL?

Entity Model

Horizontal Scalability

Simple Availability

Dynamic SchemaFlexible Schema

Simple “sharding”Manage as one

Trade consistency for availabilityHA that “just works”

What About “Big Data”

“Big” is a misnomerVolume, Variety, Velocity, Complexity

DriversCost of data acquisition $0Cost of data storage dramatically reduced“Save everything” tipping point

About turning ambient data into value

Signal Data Information

Knowledge

Insight

Action

Characteristics of Big Data

Hadoop

Hadoop for MicrosoftMany wanted to get from Hadoop into Microsoft Information PlatformExamples:• Yahoo – Populating 20+ TB Analysis Services cube from Hadoop• Klout – Creating social relevance scoring with Hadoop and SQL Analysis Services [1]Data warehousing market demanded a Hadoop solutionWanted a great port of Hadoop for Windows

The goal?Work closely with Apache community and key vendorsCreate great Hadoop experience on WindowsContribute to Hadoop ecosystem – e.g. JavaScript Map/Reduce

[1] http://corp.klout.com/blog/2011/11/big-data-bigger-brains/

Hadoop Architecture

Hadoop

demo

New data and experiencesData

Creation/Consumpti

on

DataSources

Blob Storage

SQL Azure Database

Table Storage

PC Phone Tablet Search

© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a

commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.