Clemens Vasters (@clemensv)Principal Technical Lead / Architect Windows Azure Application Platform Team – Service Bus

Building Scalable and Robust Solutions with Service Bus in Cloud and Server

Service Bus Scope

Queues Topics Relays Notifications

PCAppliances Phones & Tablets

Team SharePoint siteLOB Apps

SaaS Cloud app

Event Aggregator . . . .

• Cloud- or Self-Hosted Messaging Middleware• Windows Azure Service Bus• All Windows Azure Datacenters• Multiple Clusters (Scale-Units) per Datacenter

• Service Bus for Windows Server 1.0• Downloadable component for Windows Server (and Windows client for Dev)• Runs on top of SQL Server Express (up to SQL 2012 Always-On Cluster)• Messaging largely symmetric with cloud version, but lags up to 12 months behind

• Key Features: • Pub/Sub Reliable Messaging, Relay*, Notifications**

What is Service Bus?

*cloud-only for now**cloud-only for the forseeable future

• Building cloud services• Decoupled architecture to support scale and resilience • Support for common PubSub patterns• Advanced messaging features

• Enable Hybrid configurations• Allow secure access to local services and data (location transparency from behind

the firewall)• Rich auth model integrated with Windows Azure Active Directory• Private and Public cloud symmetry with Service Bus 1.0 for Windows Server

• Connected client scenarios• Support applications that need to notify devices (Windows8, IOS etc.) • Broadcast at scale with directed notifications and target-specific customization• Messaging libraries available in .NET, WinRT, Node.js, Java, PHP etc.

Key customer scenarios

Service Bus QueuesGateway Role Halo Backend Role

XBox

XBox

XBox

XBox

Xbox Live

Devices

Game Stats

Devices

Leaderboard

1. Game scores fan-in to Queues2. Sessions used to correlate games3. Scheduled messages to track completion4. Halo backend produces finalized stats

Service Bus Queues

POS

Marketing

IT Admin

Retail Services ISV

Sales data fan-in to QueuesMessages processed and appropriate messages sentTopics used to support Publish-Subscribe for messagesReal-time monitoring as well as decoupled inventory updatesPOS

POS

POS

Retail Web Services

Service Bus Topics

Policy Updates Remote Managemen

t

Central Repository

ISV Private Cloud

Enterprise Private Cloud

Service Bus Relay

• ISV Suite offers a turn-key solution for security & management of BYOD

• Deployed on enterprise networks behind firewalls/NATs• Uses Service Bus Relay to push policy updates/templates

and perform remote management of the Suite

Enterprise Services ISV

ISV Suite

APNS

Win Phone 8

iOS

Windows 8

CRM Online

Service Bus Notification Hub

CRM Online Service

WNS

1. Devices Authenticate with CRM2. Registration sent to Notification Hub3. Message initiated by CRM Online Service4. Customized Notifications to all devices

Filter w/ Tags Broadcast

Scale

Templates

Relay

Expose Web Services from/to anywhere• Key capabilities• Relayed One-Way Unicast and

Multicast• Relayed WCF NET.TCP with

Direct Connect Option• Relayed WCF HTTP w/ support

for REST and SOAP• Endpoint protection with

Access Control

• Coming soon• IIS Hybrid Cloud Bridge WCF

NET.TCP

WCF Plain HTTPSOAP/HTTP

HTTP, SOAP/HTTP, or NET.TCP Clients

IIS Hybrid Cloud

Bridge*

Relay Programming Model

• WCF Programming Model• Bindings functionally symmetric with WCF• WebHttpRelayBinding (HTTP/REST)• BasicHttpRelayBinding (SOAP 1.1)• WS2007HttpRelayBinding (SOAP 1.2)• NetTcpRelayBinding (Binary transport)

• Special Service Bus Bindings• NetOnewayRelayBinding (one-way)• NetEventRelayBinding (Multicast one-way)

• Transport binding elements for custom bindings

Messaging

Service Bus Brokered MessagingQueues• Ordered Message Log

• Ordered by arrival (broker assigns strongly monotonic sequence number to each msg)

• Timestamped on arrival (authoritative clock)

• Stored in SQL; once accepted, the message is as safe as any data in the SQL deployment

• Multiplexing with sessions• Two modes: Pull and Forward

• Pull – delivers messages on request, allowing concurrent readers

• Forward – delivers message to single forward destination

Topics• All features of Queues, plus• Multiple, Independent Subscriptions

• Named, durably created (optional)• Quota: Up to 2000 concurrent subscriptions• Reco is to start partitioning much, much earlier

(more later)• Rules with filter conditions on msg headers• Quota: Up to 2000 rules per subscription• Optional action set/modify message props• Each rule hit yields a message

• Topic 'tail' and subscription 'head' are fully protocol compatible with Queues

Programming Model Options

Apps

NetMessagingBinding

.NET Service Bus Messaging API

.NET WCF Service Model

SB Messaging Protocol(net.tcp, proprietary)

AMQP 1.0

Service Bus

C/C++(incl

Embedded)

Apache Proton-C

Windows Whatever

HTTP(S)

PHPclient

Node.js

client

Python

client

Java/JMS

Apache Qpid JMS AMQP 1.0

Any HTTPclient

Connectivity Options• AMQP 1.0• OASIS standard, growing base of clients across many platforms• AMQPS TCP 5671, AMQP TCP 5672• Preview status, general availability later this year

• "SBMP" Service Bus Messaging Protocol• Only supported by .NET client, Microsoft proprietary, based on

WCF's NetTcp• TCP port 9354 or HTTP tunneling over port 443 or port 80

• HTTPS• TCP 443• RFC2616 – majority of Service Bus features available for any HTTPS

client

• OASIS announced the ratification of the AMQP 1.0 Standard

• Software vendors and end-users can bet on AMQP 1.0 knowing it’s a stable, well-supported protocol standard

• The culmination of several years effort by more than 20 companies

• Technology vendors: Axway Software, Huawei Technologies, IIT Software, INETCO Systems, Kaazing, Microsoft, Mitre Corporation, Primeton Technologies, Progress Software, Red Hat, SITA, Software AG, Solace Systems, VMware, WSO2, Zenika. User firms: Bank of America, Credit Suisse, Deutsche Boerse, Goldman Sachs, JPMorgan Chase.

OASIS AMQP 1.0 Standard

• Open, standard messaging protocol• Enables cross-platform apps to be built using brokers,

libraries and frameworks from different vendors

• Features• Reliable – fire-and-forget to reliable, exactly-once delivery• Flexible – client-client, client-broker, and broker-broker

topologies• Broker-model independent – no requirements on broker

internals

Advanced Message Queuing Protocol 1.0

AMQP 1.0• Applicability• Reliability• Interoperability•Ubiquity

© JPM

org

an

Ch

ase

& C

o.

Applicability• Simple abstraction for all messaging patterns• Point to point, Pub / Sub, Request / Response

• Symmetric peer-to-peer protocol• Also enables Brokerless messaging

• Mappings to standard APIs (JMS, WCF, SOAP)• Designed for high performance• No inherently synchronous primitives• Allows aggressive pipelining• Low message overhead

Reliability• Comprehensive reliability semantics• at most once• at least once• exactly once delivery

• Recovery defined for all failure cases• Well defined error model• fail fast, then recover

Sender

Receiver

Replay Buffer

Idempotence Barrier

Ubiquity• Defines a vocabulary for messaging• Protocol consists of just nine “performatives”• Small number of key concepts (Links, Sources/Targets, Messages,

Settlement)

• Mapping to multiple messaging models• Existing messaging products implement AMQP 1.0• Stable core protocol• Guarantee compatibility with future versions• Allow for hardware implementations

Interoperability• Demonstrate interoperability

today• Different languages, different platforms,

different vendors

• Layered architecture allows extension without changing the core protocol

• AMQP 1.0 preview available today in Windows Azure Service Bus

• Service Bus .NET preview client library available• NuGet: ServiceBus.Preview

• Support for a range of third-party client libraries

• More platforms will be supported as libraries become available

• General availability in H1 2013

Service Bus AMQP 1.0 preview

Message Model• Brokered messaging properties

are not SOAP headers• System properties have special

meaning to the broker or or set by the broker

• Properties are key/value pairs that may very well carry payloads

• It's not uncommon to have messages with empty message bodies

• Message bodies are useful for a single opaque payload not exposed to the broker (e.g. encrypted content)

64KB max

256 KBmax

Service Bus Message

Body

Custom Properties

Key Value

Key Value

Key Value

Key Value

Body (Bag of Bytes)

Sys PropertiesKey Value

Key Value

Key Value

Delivery Options• Receive and Delete• Fastest. Message lost if

receiver crashes or transmission fails.

• Peek Lock• Message is locked when

retrieved. Reappears on broker when not deleted within lock timeout.

• Session + Peek Lock• Message is locked along

with all subsequent messages w/ same session-id ensuring order

Broker RReceive and Delete

Broker R

2. Delete/Unlock

1. Peek/Lock

Receive Operations• Credit-Based Model• AMQP and SBMP• Each receive operation is registered on the session and serviced in

order• Multiplexing; hundreds of receive credits can be registered

concurrently• Receive operations can be pending as long as connection is

maintained• Session/connection is maintained independently

• Long-Polling Model• Used with HTTP• Single receive operation with maximum 60 second timeout• Returns empty if no messages become available within timeout

Service Pricing*• USD $0.01 per 10,000 message transactions• Sending a message of up to 64KByte size = 1 tx• Each further 64KB segment = 1tx

• Receive a message of up to 64KByte in size = 1tx• Each long-polling operation (60s timeout) even if yielding no message = 1tx

• Outbound Windows Azure traffic surcharges (ext. receivers)• Only affects outbound traffic, does not affect intra-datacenter traffic• Zone 1: $0.05-$0.12 per GByte, Zone 2 (Asia DCs): $0.12-$0.19 per

GByte

*February 2013

Service Bus 1.0 for Windows Server

On-premises messaging scenarioWhy & What

On-Premises pub-sub

•Enterprise applications developed and deployed on-premise.•Scale ; HA ; Manageability ; Rich messaging feature set

Develop On-premise •High fidelity develop, test, debug experience•DevBox deployment (HW&SF pre-reqs) ; Tools (VS); debug mode ; symmetry

Flexible Deployment •Driven by enterprise desire for deployment choice (or perception thereof)•Full symmetry ; Ease of migration

Tiered Deployment •Maintain control of mission critical data / systems•Unable/unwilling to migrate legacy systems•Gradual migration to cloud•Message Federation ; Store-and-Forward

Deploy

DeployDeploy

DataApp

Develop

Develop&Deploy

Node

Node

Node

• Clustered deployment

• Identical nodes• Stateless gateways

• Handles incoming client requests.

• Routes request to backend.

• Backend• Load one or more containers.

• Databases• Database holds messages +

metadata.• One database per container.

Service Bus Server – Architecture

SB GatewaySB Messaging

Backend Message Store

SB Messaging

Backend

SB Messaging

Backend

Message Store

Message Store

Message Store

Gateway DB

SB Gateway

SB Gateway

Service Bus Farm Topologies

Windows Server 2008 R2 SP1 Windows Server 2012

DevBox: Windows 7 64-bit, Windows 8 64-bit

SQL Server 2008 R2 SP1 SQL Server 2012

DevBox: SQL Express 2008 R2 SP1 or higher

Web PI-based

Service Bus Deployment - Install

Service Bus Platform - PerformanceHardware3-node farm + 1 SQL serverAll machines are equipped with 4x quad-core CPU @ 2.3GHz, 24GB RAM.SQL server equipped with 6 disks.

Measured performance for flow ratesSingle queue throughput (1KB): 20,000 messages/s (40,000 operations/s)Single topic with 1 subscription (1KB): 14,200 messages/s (28,400 operations/s)Single topic with 1000 subscriptions (1KB): 26 messages/s (26026 operations/s)Latency: 6msBottleneck: disk I/O

Always able to get more by using faster machines.

Messaging Patterns

Brokered Transfer

• Load Leveling• Receiver receives and processes at its own pace. Can never be

overloaded. Can add receivers as queue length grows, reduce receiver if queue length is low or zero. Gracefully handles traffic spikes by never stressing out the backend.

• Offline/Batch• Allows taking the receiver offline for servicing or other reasons.

Requests are buffered up until the receiver is available again.

S RQueue

Competing Consumer

• Load Balancing• Multiple receivers compete for messages on the same queue (or

subscription). Provides automatic load balancing of work to receivers volunteering for jobs.

• Observing the queue length allows to determine whether more receivers are required.

S RQueue

R

R

Fan-In

• Concentrator• Fan information into a single queue from a range of data sources

• Multi-Stage Aggregration / Rollup• Fan into a set of queues, perform aggregation/roll-up/reduction and

fan further.

S RQueue

S

S

Taps and Fan-Out

• Message Distribution• Each receiver gets its own copy of each message. Subscriptions are

independent. Allows for many independent 'taps' into a message stream. Subscriber can filter down by interest.

• Constrained Message Distribution (Partitioning)• Receiver get mutually exclusive slices of the message stream by

creating appropriate filter expressions.

S RTopic SubSub

Sub

R

R

Correlation

• Correlation is required to set up reply paths between sender and receiver.

• Three correlation models in Service Bus: Message-correlation, subscription-correlation, session-correlation

Broker

S RSend Queue

Reply Queue

Broker Message

Body

PropertiesRepl

yQueue Name

Id {id}

{id}

Broker MessageProperties

C-Id {id}

N-to-M With Correlation

Broker

S

RSend Queue

Reply Queue

Broker Message

Body

PropertiesRepl

yQueue Name

Id {id}

{id}

Broker MessageProperties

C-Id {id}

S

Reply Queue

{id}

Broker MessageProperties

C-Id {id}

R

R

Backe

nd

Sca

le-O

ut

var sender = QueueClient.CreateFromConnectionString(cs, "test");sender.Send(new BrokeredMessage { Label = "Hello" });

// Send a message with system and application properties setvar bm = new BrokeredMessage();bm.Label = "PurchaseOrder123";bm.TimeToLive = new TimeSpan(0, 5, 0);bm.Properties["PurchaseOrderID"] = 1234;sender.Send(bm);

Sending Messages

var receiver1 = QueueClient.CreateFromConnectionString(cs, "test", ReceiveMode.ReceiveAndDelete);var msg1 = receiver1.Receive();

var receiver2 = QueueClient.CreateFromConnectionString(cs, "test", ReceiveMode.PeekLock);var msg2 = receiver2.Receive();try{ // Process msg2 msg2.Complete();}catch { msg2.Abandon(); throw;}

Receiving Messages

• Filter conditions operate on message properties

• SQL-derived syntax• Implicit conversion• Comparison• Logic, Arithmetic

Operators• String Concatenation• LIKE string pattern

matching• Set matching

Topics: SQL Subscription Filters

var filter = new SqlFilter("InvoiceTotal < 10000");

<predicate ::= { NOT <predicate> } | <predicate> AND <predicate> | <predicate> OR <predicate> | <expression> { = | <> | != | > | >= | < | <= } <expression> | <property> IS [NOT] NULL | <expression> [NOT] IN ( <expression> [, ...n] ) | <expression> [NOT] LIKE <pattern> [ESCAPE <escape_char>] | EXISTS ( <property> ) | ( <predicate> )

<expression> ::= <constant> | <property> | <expression> { + | - | * | / | % } <expression> | { + | - } <expression> | ( <expression> )

• Actions operate on and modify message properties when the filter evaluates to 'true'

• SQL-derived syntax• SET to create/modify

props• REMOVE to remove props• Can reference existing

props• Arithmetic and string

concat• Newid() to create unique

IDs

Topics: SQL Subscription Actions<statements> ::= <statement> [, ...n] | <action> [;]

<action> ::= SET <property> = <expression> | REMOVE <property>

<expression> ::= <literal> | <function> ::= newid() | <property> | <expression> { + | - | * | / | % } <expression> | { + | - } <expression> | ( <expression> )

var rule = new SqlRuleAction("SET FullName = FirstName + ' ' + LastName");

• Chain Queues/Topics • Scale out

Topics/Subscriptions• Fan-in from several

Queues• Routing

decoupling/redirect

Auto Forwarding

QueueDescription destinationQ = new QueueDescription("myQ2");QueueDescription sourceQ = new QueueDescription("myQ1");sourceQ.ForwardTo = "myQ2";

NamespaceManager nm = NamespaceManager.Create();nm.CreateQueue(destinationQ);nm.CreateQueue(sourceQ);

T TS

S

S

T S

S

S

T

T

Q

TQ

• Scale beyond per Topic quotas

• Increased message flow rate• Flow rate is (roughly) entity

throughput divided by number of subscriptions

• Narrower, taller trees are faster

• Partition by branches• Volatile subscriptions at

edge

Fan-Out Trees

T S

S

S

T S

S

S

T

T

T

T S

S

S

T

T

T

T S

S

S

T

T

T

SSS

SSS

SSS

SSS

SSS

SSS

SSS

SSS

SSS

Group IN (‘A’, ‘B’, ‘C’, ‘D’, ‘E’)

DeviceId = ‘123’ OR UserId = ‘clemensv’

OR Group IN (‘A’, ‘B’, ‘C’)

DeviceId = ‘456’ OR UserId = ‘clemensv’

OR Group IN (‘A’, ‘B’, ‘C’)

• Handle extreme bursts of traffic• End-of-period processing (hourly, daily, etc.)

• Partition clients across and pin them to ingress queues by some criterion

• Total ingress flow rate capacity multiplied• All throughput limits are per socket and per

queue

• Ingress bursts forwarded into concentrator queues per auto-forward for continuous processing

Fan-In Trees

Q

Q

Q

Q

Q

Q

Q

Q

Q

Binary Protocols Matter with MessagingHTTP SBMP/AMQP

Client Client

Q S

T

N Q S

T

N

Socket Socket

HTTP 1 Entity per Socket 1 Pending Operation per Socket 60s operation timeout (NAT/Prx)SBMP/AMQP Unlimited Multiplexed Entities and Unlimited Pending Ops per Socket No fixed operation timeout Session Support (coming in AMQP)

SignalR• Lightweight Signaling Library for Web Clients in

ASP.NET team• Transport abstraction• Simple programming abstraction via “Hubs”• Ephemeral publish/subscribe model

• Multiple server-to-client transport models:• Web Sockets, SSE, Forever-Frame, Long Polling• Auto-detects best transport option• “bring-your-own” security model

• Achieves very high density of 10,000s of concurrent clients per node

SignalR Architecture

IMessageBus

Connections

Long Polling, Web Sockets, Server-Side Events, Forever-Frame

Browsers

Key Value

FilteredReceive

key1key2

…keyN

SignalR and Service Bus

IMessageBusIMessageBus

Node A Node B

Topic A B

Topic A B

Topic A B

Client Pinning

Throughput Scale-Out

Composite Cursor

Node.js/Socket.IO and Service Bus

Socket.ioSocket.io

Node A Node B

Topic A B

Topic A B

Topic A B

Throughput Scale-Out

Notification Hubs

• Increase app engagement• Notify the user of some event, e.g.• Updating a tile on a Windows Store app with current financial/weather information• Alerting a user with a toast that some work item has been assigned• Displaying a badge of the number of current sales leads in a CRM app

• Push notifications require a platform specific service• Each platform (Windows Store, iOS, Android, Windows Phone) has a different push

notification service• Different capabilities and protocols

• An e2e solution requires lots of back-end code• Back-end has to:• Store and keep up to date the device information• Implement platform-specific protocols

Push notifications

• Registration at app launch• Client app contacts Platform

Notification Service, to retrieve current device/channel handle (e.g. ChannelURIs, device tokens)

• App updates handle in back-end

• Sending Notification• App back-end send notification to the

PNS• PNS pushes the notification to the

app on the device

• Maintenance• Delete expired handles when PNS

rejects them

Push notification lifecycle

PlatformNotification

Service

App back-end

Client app

• Platform dependency• Different communication protocols to PNS’ (e.g. HTTP vs TCP, xml payload vs json

payload)• Different presentation formats and capabilities (tiles vs toasts vs badges)

• Routing• PNS’ provide a way to send a message to a device/channel• Usually notifications are targeted at users or interest groups (e.g. employees

assigned to a customer account)• App back-end has to maintain a registry that associates device handles to interest

groups and users

• Scale• App back-end has to store current handles for each device

At high scale (millions of devices) results in high storage and computation costs• Broadcast to millions of devices with low latency requires extensive parallelization

(both DBs ad VMs)

Challenges of push notifications

• One-time set up• Create a Notification Hub in Service

BusIt contains the credentials required by the Platform Notification Service (PNS)

• Register• The client app retrieves its current

handle from the PNS• Client app creates (or updates) a

registration on the Notification Hub with the current handle

• Send Notification• The app back-end sends a message

to the Notification Hub• Notification Hub pushes it to the

PNS’

Using Notification Hubs

APNs WNS

Service Bus Notification Hub

App back-end

iOS app Windows Storeapp

• No platform-specific protocols• App back-end just communicates with the Notification Hub.

• Avoid storing device information in the app back-end• Notification Hub maintains the registry of devices and the associations

to users/interest groups

• Broadcast• Push notifications to millions of devices (across platforms) with a single

call

Advantages of using Notification Hubs

• Tags as interest groups• Client app can register with a

set of tags• Tags are simple strings (no

pre-provisioning is required)• App back-end can target all

clients with the same tag

• You can use tags also for• Multiple type of interest

groups, e.g.• Follow bands: tag

“followband:Beatles”• Follow users: tag

“followuser:Alice”• Tag devices with a user id

Using tags to route messages

Service Bus Notification HubApp back-

end

Tag:”Beatles”

Tag:”Wailers”

Tag:”Beatles”

• Registration• Client apps can register with a

platform specific template, e.g. • Alice’s Surface registers with Windows

Store ToastText01 template• Bob’s iPhone with the Apple JSON

template:{ aps: {alert: “$(message)”}}

• Send notification• App back-end sends a platform

independent message: {message: “Hello!”}

• Support for native• Notification Hubs also supports

platform payloads sent by the back-end• (but no multi-platform broadcast)

Using templates for multi-platform push

Service Bus Notification HubApp back-

end

<toast><visual><binding template=\"ToastText01\"><text id=\"1\">$(message)</text>

</binding></visual>

</toast>

{aps: {

alert: “$(message)”

}}

{ message: “Hello!” }

Hello!

Hello

!

Thank You For Listening!

© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to

be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.