SQL Server 2014

Dung T. Thoi

Drive Real-Time Business with Real-Time Insights

Over 100x query speed and

significant data compression with

In-Memory ColumnStore

Up to 30x faster transaction

processing with In-Memory OLTP

Greater

performance

with In-Memory

Analysis Services

Billions of rows

per second with

PowerPivot

In-Memory for

Excel

4

Faster InsightsIN-MEMORY ANALYTICS

Faster QueriesIN-MEMORY DW

Faster TransactionsIN-MEMORY OLTP

Analysis Services

In-Memory

In-Memory

ColumnStore

PowerPivot

In-Memory for Excel

The Journey to Complete In-Memory

PowerPivot

In-Memory for Excel

20122008 R2

Enhanced In-Memory

ColumnStore

In-Memory OLTP

Analysis Services

In-Memory

PowerPivot

In-Memory for Excel

In-Memory in

the cloud

2014

5

Key Trends impacting In-Memory Design

Decreasing Cost of Memory$/GB of Memory

1

10

100

1 000

10 000

100 000

1 000 000

19

90

19

91

19

92

19

93

19

94

19

94

19

95

19

96

19

97

19

98

19

99

20

00

20

00

20

01

20

02

20

04

20

05

20

07

20

08

20

09

20

11

US$

/GB

CPU Performance Flattening Out

0

1

10

100

1 000

10 000

100 000

1 000 000

10 000 000

1970 1975 1980 1985 1990 1995 2000 2005 2010

7

SQL Server integration

• Same manageability,

administration, and

development experience

• Integrated queries and

transactions

• Integrated HA and

backup/restore

Main-memory

optimized

• Optimized for in-memory

data

• Indexes (hash and range)

exist only in memory

• No buffer pool

• Stream-based storage for

durability

High concurrency

• Multiversion optimistic

concurrency control with full

ACID support

• Core engine uses lock-free

algorithms

• No lock manager, latches, or

spinlocks

T-SQL compiled to

machine code

• T-SQL compiled to machine

code via C code generator

and Visual C compiler

• Invoking a procedure is just

a DLL entry-point

• Aggressive optimizations at

compile time

Steadily declining memory

price, NVRAMMany-core processors Stalling CPU clock rate TCO

Hardware trends Business

Hybrid engine and

integrated experience

High-performance data

operations

Frictionless scale-up Efficient, business-logic

processing

Benefits

In-M

em

ory

OLT

P T

ech

Pill

ars

Drive

rs

In-Memory OLTP Architecture

DemoIn Memory OLTP

Main-memory

optimized

• Optimized for in-memory

data

• Indexes (hash and ordered)

exist only in memory

• No buffer pool

• Stream-based storage for

durability

Steadily declining memory

price, NVRAM

Hardware trends

Table constructsFixed schema; no ALTER TABLE; must drop/recreate/reload

No LOB data types; row size limited to 8,060

No constraints support (primary key only)

No identity or calculated columns, or CLR

Data and table size considerationsSize of tables = (row size * # of rows)

Size of hash index = (bucket_count * 8 bytes)

Max size SCHEMA_AND_DATA = 512 GB

IO for durability SCHEMA_ONLY vs. SCHEMA_AND_DATA

Memory-optimized filegroup

Data and delta files

Transaction log

Database recovery

Design Considerations For Memory-optimized Tables

10

High performance data

operations

Benefits

In-M

em

ory

OLT

P Te

ch P

illars

Drive

rs

Create Table DDL

CREATE TABLE [Customer](

[CustomerID] INT NOT NULL

PRIMARY KEY NONCLUSTERED HASH WITH (BUCKET_COUNT = 1000000),

[Name] NVARCHAR(250) NOT NULL

INDEX [IName] HASH WITH (BUCKET_COUNT = 1000000),

[CustomerSince] DATETIME NULL

)

WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA);

This table is

memory optimizedThis table is durable

Secondary indexes

are specified inline

Hash index

High concurrency

• Multiversion optimistic

concurrency control

with full ACID support

• Core engine uses

lock-free algorithms

• No lock manager,

latches, or spinlocks

Many-core processors

Hardware trends

12

Frictionless scale-

up

Benefits

In-M

em

ory

OLT

P Te

ch P

illars

Drive

rs

Impact of no locks or latchesWrite-write conflict: design application for condition with try.catch

Applications dependent on locking; may not be a good candidate

Multiple versions of recordsIncreases the memory needed by memory-optimized tables

Garbage Collection used to reclaim old versions

Transaction scopeSupport for Isolation Levels: Snapshot, Repeatable Read, Serializable

Commit time validation; again must retry logic to deal with failure

High Concurrency Design Considerations

Example: Write Conflict

13

Time Transaction T1 (SNAPSHOT) Transaction T2 (SNAPSHOT)

1 BEGIN

2 BEGIN

3 UPDATE t SET c1=‘bla’ WHERE c2=123

4 UPDATE t SET c1=‘bla’ WHERE

c2=123 (write conflict)

First writer

wins

T-SQL compiled to

machine code

• T-SQL compiled to

machine code via C code

generator and Visual C

compiler

• Invoking a procedure is

just a DLL entry-point

• Aggressive optimizations

at compile-time

Stalling CPU clock rate

Hardware trends

Design considerations for native compiled stored procedures

14

Efficient business-logic

processing

Benefits

In-M

em

ory

OLT

P T

ech

Pill

ars

Drive

rs

Native compiled stored

procedures

Non-native compilation

Performance High. Significantly less

instructions to go through

No different than T-SQL

calls in SQL Server today

Migration strategy Application changes;

development overhead

Easier app migration as can

still access memory-

optimized tables

Access to objects Can only interact with

memory-optimized tables

All objects; access for

transactions across memory

optimized and B-tree tables

Support for T-SQL

constructs

Limited T-SQL surface area (limit on

memory-optimized

interaction)

Optimization, stats, and

query plan

Statistics utilized at CREATE

-> Compile time

Statistics updates can be

used to modify plan at

runtime

Flexibility Limited (no ALTER

procedure, compile-time

isolation level)

Ad-hoc query patterns

Create Procedure DDL

CREATE PROCEDURE [dbo].[InsertOrder] @id INT, @date DATETIME

WITH

NATIVE_COMPILATION,

SCHEMABINDING,

EXECUTE AS OWNER

AS

BEGIN ATOMIC

WITH

(TRANSACTION

ISOLATION LEVEL = SNAPSHOT,

LANGUAGE = 'us_english')

-- insert T-SQL here

END

This proc is natively

compiled

Native procs must be

schema-bound

Atomic blocks

• Create a transaction

if there is none

• Otherwise, create a

savepoint

Execution context is

required

Session settings are fixed

at create time

Memory-optimized Table FilegroupData Filegroup

SQL Server.exe

Hekaton Engine: Memory_optimized Tables &

Indexes

TDS Handler and Session Management

Hekaton Integration and Application Migration

Natively Compiled SPs

and Schema

Buffer Pool for Tables & Indexes

Proc/Plan cache for ad-hoc T-SQL and

SPs

Client App

Transaction Log

Query

InterOp

Non-durable Table

T1 T4T3T2

T1 T4T3T2

T1 T4T3T2

T1 T4T3T2

Tables

Indexes

Interpreter for TSQL, query plans,

expressions

T1 T4T3T2

T1 T4T3T2

Checkpoint & Recovery

Access Methods

Parser,

Catalog,

Algebrizer,

Optimizer

Hekaton

Compiler

Hekaton

Component

Key

Existing SQL

Component

Generated .dll

17

Integrated Experience

Backup and restoreFull and log backup and restore is supported; piecemeal

restore is supported

Failover clusteringFailover time depends on size of durable memory-optimized

tables

AlwaysOnSecondary has memory-optimized tables in memory

Failover time is not dependent on size of durable memory-

optimized tables

DMVs, catalog views, performance monitor

counters, XEventsMonitoring memory, garbage collection activity, and

transaction details

SQL Server Management Studio (SSMS)Creating, managing, and monitoring tables; databases and

server

In-Memory DWColumnStore

C

1

C

2

C

3

C

5

C

6

C

4Columnstore Index Representation

In-Memory In the Data WarehouseData Stored Row-Wise: Heaps, b-Trees, Key-Value

20

• In-Memory ColumnStore

• Both memory and disk

• Built-in to core RDBMS engine

• Customer Benefits:

‐ 10-100x faster

‐ Reduced design effort

‐ Work on customers’ existing hardware

‐ Easy upgrade; Easy deployment

“By using SQL Server 2012 In-Memory

ColumnStore, we were able to extract about

100 million records in 2 or 3 seconds versus

the 30 minutes required previously. “

- Atsuo Nakajima Asst Director, Bank of Nagoya

Traditional Storage ModelsData Stored Row-Wise: Heaps, b-Trees, Key-Value

21

• Relational, dimensional, map reduce

…

ColumnStore

Jacques Dupont M

Jeanne Dubois F

Paul Dupuis M

Jean Dutoit M

Jacques Durand M

Jeanne Dupont F

Isabelle Dupuis F

Alex Dumont M

In-Memory DW Storage ModelData Stored Column-wise

23

• Each page stores data from a single column

• Highly compressed

‐ More data fits in memory

• Each column can be accessed independently

‐ Fetch only columns needed

‐ Can dramatically decrease I/O

C1 C2 C3 C5 C6C4

DemoIn Memory ColumnStore

In-Memory DW: Clustered & Updatable

26

• Fast execution for data warehouse queries

‐ Speedups of 10x and more

• No need for separate base table

‐ Save space

• Data can be inserted, updated or deleted

‐ Simpler management

• Eliminate need for other indexes

‐ Save space and simpler management

• More data types supported

Comparing Space Savings101 Million Row Table + Index Space

27

19,7GB

10,9GB

5,0GB4,0GB

6,9GB

1,8GB

TABLE WITH

CUSTOMARY

INDEXING

TABLE WITH

CUSTOMARY

INDEXING (PAGE

COMPRESSION)

TABLE WITH NO

INDEXING

TABLE WITH NO

INDEXING (PAGE

COMPRESSION)

TABLE WITH

COLUMNSTORE

INDEX

CLUSTERED

COLUMNSTORE

Structure of In-Memory DWHow It Works

28

• CREATE CLUSTERED COLUMNSTORE

‐ Organizes and compresses data into columnstore

• BULK INSERT

‐ Creates new columnstore row groups

• INSERT

‐ Rows are placed in the row store (heap)

‐ When row store is big enough, a new columnstore row group is created

ColumnStoreDeleted

Bitmap

Row Store

Partition

Structure of In-Memory DWHow It Works (cont'd)

• DELETE

‐ Rows are marked in the deleted bitmap

• UPDATE

‐ Delete plus insert

• Most data is in columnstore format

ColumnStoreDeleted

Bitmap

Row Store

Partition

Batch Mode ProcessingWhat’s New?

30

• SQL Server 2014

‐ Support for all flavors of JOINs

OUTER JOIN

Semi-join: IN, NOT IN

‐ UNION ALL

‐ Scalar aggregates

‐ Mixed mode plans

‐ Improvements in bitmaps, spill support, …

AlwaysOn Enhancements

Availability Groups in SQL Server 2012

SQL Server 2014: max: 8 secondary's – 2 synchronus

Replace Database Mirroring and Log Shipping with Availability Group

Primary Data Center

Synchronous / Asynchronous

Disaster Recovery Data

Center

Primary Secondary

Fileshare Witness

Synchronous

Secondary

Windows Server Failover

Cluster

Availability

Group

Demo

AlwaysOn in SQL Server 2014

• What’s being delivered

• Increase number of secondaries from four to eight

• Increase availability of readable secondaries

• Support for Windows Server 2012 CSV

• Enhanced diagnostics

• Main benefits

• Further scale out read workloads across (possibly geo-distributed) replicas

• Use readable secondaries despite network failures (important in geo-distributed environments)

• Improve SAN storage utilization

• Avoid drive letter limitation (max 24 drives) via CSV paths

• Increase resiliency of storage failover

• Ease troubleshooting

• SQL Server 2012: Read workloads killed during network failures

• Geo-distributed environments (e.g. failure/upgrade of network equipment, ISP failures)

• Hybrid (on-premise to Azure) deployments

• SQL Server 2014: read workloads not impacted during network failures.. or primary down.. or cluster quorum loss..

• Readable secondaries remain available during “Resolving” state

• Requires direct connections to readable secondaries (Read-only routing not supported yet)

• Replica state and last commit time available in DMV/Dashboard

Availability GroupsIncreased Readable Secondaries Availability

Other New/Enhanced Features

Hybrid cloud solutions

Restore to VMGeo-replication

Backup to Microsoft Azure

Azure storage

WA

Mic

roso

ft

Azu

re B

lob

s• On-site/off-site storage costs

• Device management costsB

ox

• XDrives limited to 1 terabyte

• Max 16 drives

• Manage drives and policy

• Near “bottomless” storage

• Off-site, geo-redundant

• No provisioning

• No device management

• Media safety (decay-free)

• Remote accessibility

Demo

Hybrid cloud solutions

Typical Configuration

47

Characteristic of Cloud DR for SQL Server

48

Inverse Deployment Configuration Supported

49

AlwaysOn Replica in Microsoft Azure

50

Easy on-ramp to cloud

Deploy Database to a Azure VM

SolutionNew SSMS Wizard: Deploy Database to a Microsoft Azure VM

Demo

Complete and consistent data platform

Call to action

Download SQL Server 2014

www.microsoft.com/sqlserver

© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other

countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond

to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date

of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION