messaging patterns in the cloud

itcampro@ itcamp13# Premium conference on Microsoft technologies

Messaging Patterns in the Cloud

Radu VunvuleaiQuest Group

@RaduVunvuleahttp://vunvulearadu.blogspot.com


Private & Public CloudHuge thanks to our sponsors!


Private & Public Cloud

• World of Messages• On-Premise and Cloud solutions• Messaging Patterns • Costs, Benefits and Limitations

Agenda


WORLD OF MESSAGES


Private & Public CloudWorld of Messages


Private & Public CloudWorld of Messages

Message

Message

Message



Enterprise Service Bus

?




Interaction and communication between software applications


Private & Public CloudEnterprise Service Bus

Topic

Subscriptions


ON-PREMISE AND CLOUD SOLUTIONS



• Pro– Open Source– Flexible– Good price – Transactional queue read/write– Flexible load distribution

• Cons– No queue management– Document not up to date– Only one queue technology supported

(MSMQ)

NService Bus



• Pro– Free for commercial use– Supports MSMQ and RabittMQ– Open Source– Good administration console

• Cons– No commercial support– Not as fast as NServiceBus

MassTransit



• Pro– Scalable– Lots of features– Good administration console– Support– Cross platform and multiple environments

• Cons– Is not free– Latency

Azure Service Bus


MESSAGING PATTERNS


Private & Public CloudMessage Filter


Private & Public CloudMessage Filter

Topic

Subscriptions



• Custom rules based on message attributes

• Add attributes to message

Message Filter

BrokeredMessage msg = new BrokeredMessage(messageContent);msg.Properties["colorCode"] = 1;

• Create subscription with custom rule

RuleDescription rule = new RuleDescription(){ // 1 - green, 2 - orange Filter = new SqlFilter("colorCode = 1");}namespaceManager.CreateSubscription(

"itcamp", "greensubscription", rule);

http://www.google.com/search?q=new+msdn.microsoft.com




Private & Public CloudMessage Splitter


Private & Public CloudMessage Splitter

Topic

Subscriptions



• Group messages based on session

Message Splitter

MessageSession msgSession = msgReceiver.AcceptMessageSession();BrokeredMessage message;while ((message = msgSession.Receive()) != null){ // Process message }msgSession.Complete()

BrokeredMessage message = new BrokereMessage(content);message.SessionId = "sessionId";

• Consume messages based on session id





Message Splitter

MessageSession msgSession = msgReceiver.AcceptMessageSession();BrokeredMessage message;while ((message = msgSession.Receive()) != null){ // Process message }msgSession.Complete()


• Consume messages based on session id

Transactions Support



Private & Public CloudMessage Aggregator


Private & Public CloudMessage Aggregator

Topic

Subscriptions




Message Aggregator


• Use correlation id and filtersBrokereMessage message = new BrokereMessage(content);message.CorrelationId = “CJ"topic.Send(message);

namespaceManager.CreateSubscription( “itcamp”, “iquestsubscriptioncar”, new CorrelationFilterExpression(“CJ"));





Private & Public CloudMessage Resequencer



Topic

Subscriptions

1234

1234

1234



Topic

Subscriptions

1234

1234

3412



• Session, message index and message count

Message Resequencer

• Mark message as dead letter when in incorrect order

• Iterate dead letter queue to access a message with a lower index

BrokeredMessage message = new BrokereMessage(content);message.Properties["index"] = 2;message.Properties["count"] = 10message.SessionId = “CJ-15-IQU";




MessageSession messageSession = queueClient.AcceptMessageSession();int currentIndex = 1;while(true){ BrokeredMessage message = messageSession.Receive(); if(int.Parse(message.Properties[“index”]) != currentIndex) { message.DeadLetter(); continue; } … message.Complete(); if(int.Parse(messsage[“count”]) == currentIndex) { break; } currentIndex++;}


Private & Public CloudMessage Recipient

Topic

Subscriptions



• Use a property to specific the list of groups

Message Recipient

• Define custom filters using LIKE operator

BrokeredMessage message = new BrokeredMessage(content);message.Properties[“groups”] = “orange magenta”;

SqlFilter filter = new SqlFilter(“groups LIKE ‘%orange%’”);

topic.AddSubscription(“subscription3”, filter);

• OR different properties for each group type

SqlFilter filter = new SqlFilter(“EXISTS orange”);







• Use a property to specific the list of groups

Message Recipient

• Define custom filters using LIKE operator

BrokeredMessage message = new BrokeredMessage(content);message.Properties[“groups”] = “orange magenta”;

SqlFilter filter = new SqlFilter(“groups LIKE ‘%orange%’”);


• OR different properties for each group type

• Use only one property and a prime number for each group




Private & Public CloudContent-Based Router

Topic

Subscriptions


Private & Public CloudContent-Based Router

Topic

Subscriptions

?


Private & Public CloudScatter-Gather

Topic

Subscriptions

Queue


Private & Public CloudScatter-Gather

Topic

Subscriptions

Queue

?


Private & Public CloudDynamic Router

Topic

Subscriptions



• [key, values] properties

• Store properties

• Decorate properties

Dynamic Router


COSTS, BENEFITS AND LIMITATIONS



• What are the costs of processing:– 24kB message size– 10M messages– 1 Topic- 8 hours

Costs



• 10$ Sending • 27.46$ Bandwidth (sending)

Costs




• 10 $ Receiving• 0 $ Bandwidth (receiving)

Costs




• 10 $ Receiving• 0 $ Bandwidth (receiving)• 47.46$ Costs related to Service Bus

Costs



• 10$ Sending • 27.46$ Bandwidth (sending)• 10 $ Receiving• 0 $ Bandwidth (receiving)• 47.46$ Costs related to Service Bus

• 8.64$ 4 Medium Worker Roles used to consume messages

• 56.1$ Total cost

Costs



• Cheap • 99.9% Uptime • Extremely scalable and flexible• No message is lost• Filters and actions support• REST API• Death-letter and transaction support• Same API for on-premise and cloud

Benefits



• Not for real-time application• When processing more than 1M

messages on the same topic in a 30 minutes time interval latency increases

• You need to pay for each send/receive command

• Batch maximum size – 100 messages

Limitations


Q & A


THANK YOU

messaging patterns in the cloud

Technology

premium conference

message attributes

process message

cloud solutions

filtersbrokeremessage

new ruledescription

new sqlfiltercolorcode

free latencyazure service