cloud architectural patters and microsoft azure tools
DESCRIPTION
Cloud architectural patters and Microsoft Azure tools (a presentation at Atlanta code camp - 2014)TRANSCRIPT
![Page 1: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/1.jpg)
Welcome
Cloud Architectural Patterns (using Microsoft Azure tools)
Pushkar Chivate
![Page 2: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/2.jpg)
Agenda• Quick Look: Microsoft Azure as a platform
• Overview of patterns for cloud applications
• Specifically Data-Management, Resiliency and Messaging patterns
• Demo • Azure Tables,
• DocumentDB,
• Azure Service bus (Queues and Pub-Sub architecture)
![Page 3: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/3.jpg)
What is the Cloud?
IT
Capacity
Time
Load
Forecast
Allocated
IT-capacity
Under capacity
Over
capacity
Actual
Load
![Page 4: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/4.jpg)
How the Cloud Helps?
IT
Capacity
Time
Load
Forecast
Actual
Load
Capacity on
Demand
![Page 5: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/5.jpg)
Cloud ModelsIaaS Infrastructure as a service(e.g. Azure VMs)
PaaS Platform as a service(e.g. Azure Storage, Service Bus etc.)
SaaS Software as a Service(e.g. Office 365)
![Page 6: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/6.jpg)
Microsoft Azure is a cloud platform
● On-Demand
● Self-service
● Pay per use
● Highly automated
● Designed for Failure
● Designed for Scale
![Page 7: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/7.jpg)
Microsoft AzureDesigned for Failure
Network load balancer
VM1 VM2 VM3 VM4 VM5 VM6
VM7 VM8
Fault Domain
1
Fault Domain
2Fault Domain
3UD1
UD2
![Page 8: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/8.jpg)
Microsoft AzureDesigned for Scale
Scale out
Network load balancer
VM1 VM2 VM3 VM4
![Page 9: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/9.jpg)
Azure VMs, Websites&
Rethinking AppsA look at various features
Azure offers…
![Page 10: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/10.jpg)
Architectural Pattern
An architectural pattern is a general, reusable solution to a commonly occurring problem within a given
context.
![Page 11: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/11.jpg)
Cloud PatternsData-Management Patterns
● Index Table & Sharding PatternCreate indexes over the fields in data stores that are frequently referenced by query criteria. This pattern can improve query performance by allowing applications to more quickly retrieve data from a data store. Divide a data store into a set of horizontal partitions shards
● Materialized View PatternGenerate prepopulated views over the data in one or more data stores when the data is formatted in a way that does not favor the required query operations. This pattern can help to support efficient querying and data extraction, and improve performance
● Command and Query Responsibility Segregation (CQRS) PatternSegregate operations that read data from operations that update data by using separate interfaces.
![Page 12: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/12.jpg)
Cloud PatternsMessaging Patterns
● Pipes and Filter PatternDecompose a task that performs complex processing into a series of discrete elements that can be reused.
● Priority Queue PatternPrioritize requests sent to services so that requests with a higher priority are
received and processed more quickly than those of a lower priority
.
● Queue-based Load Leveling PatternUse a queue that acts as a buffer between a task and a service that it invokes in order to smooth intermittent heavy loads that may otherwise cause the service to fail or the task to timeout.
![Page 13: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/13.jpg)
Data StorageVarious data storage techniques
● SQL Azure
● Azure Table
● Blob storage
● Document DB
![Page 14: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/14.jpg)
Materialized View PatternThis pattern can be used to support efficient querying and data extraction, and improve
application performance
What is a materialized view?
Materialized view is a database object that contains results of a query
![Page 15: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/15.jpg)
Materialized View
OrderID Account
1 A
2 B
ItemID Name Stock
100 Shirts 100
101 Pants 125
OrderID ItemID Qty Ordered
1 100 2
1 101 3
2 100 2
Materialized View
Database
tables
ItemID Name Total Qty Ordered
100 Shirts 4
101 Pants 3
Denormalized data
Data refreshes
periodically
![Page 16: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/16.jpg)
Materialized Views Vs Regular Views
Materialized View● Stored as a physical
database object
● Indexes on the base tables are not used , can create indexes on materialized view
● Faster performance
● Needs to be updated
Regular View● Stored as a query
against the base object
● Indexes on the base tables are used
● No performance improvement
● Need not be updated
![Page 17: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/17.jpg)
Materialized view
When underlying data is complex and difficult to query
directly
Simplify queries - expose data in a way that doesn’t
require knowledge of underlying data structure
Provide access to specific subset of data (security
reasons)
Bridging the disjoint when using different data stores
When to use materialized
views?
![Page 18: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/18.jpg)
Materialized viewSnapshot
s
Database
1
Table
1
Materialized view
1
Database
2
Table
1
Materialized view
2
Snapsho
t
Materialized
view
![Page 19: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/19.jpg)
Databases supporting Materialized views
Oracle
Called Indexed Views
IBM DB2SQL
Server
MySQL
No Native support
Can be implemented
using triggers
Called Materialized Query
Table
Called Materialized View
![Page 20: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/20.jpg)
Materialized view
When not to use materialized views?● The source data is simple and easy to query.
● The source data changes very quickly, or can be accessed without using a view. The processing overhead of creating views may be avoidable in these cases.
● Consistency is a high priority. The views may not always be fully consistent with the original data
![Page 21: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/21.jpg)
Azure Tables
What are Azure Tables?
● Non relational entities (not RDBMS).
● Tables are nothing but collection of entities
● Entities are comprised of properties
● Properties are name value pairs
![Page 22: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/22.jpg)
Azure TablesAzure Storage
Account
Employees
(Table)
Entity
Property
Entity
Property
Property PropertyProperty
![Page 23: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/23.jpg)
Azure Table EntitiesAzure Storage
Account
Employees
(Table) Entity
Property
<Name,Value
>
Property
<Name,Value
>
Entities● Partition Key
● Row Key
● Timestamp
● … other properties
● Each entity can have different number and type of properties
Partition Key + Row Key = Unique Identifier
(No support for secondary index in Azure Table)
Property
<Name,Value
>
![Page 24: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/24.jpg)
Azure Tables Partitions
Employees (Table) Certifications (Table)
(Entity)
EmployeeName=…CertName=…BirthDate=…
CertNumber=…
EmployeeName=…CertName=…BirthDate=…
CertNumber=…
FavoriteTeam=…
(Entity)
CertName=…EmployeeName=…
CertNumber=…BirthDate=…
CertName=…EmployeeName=…
CertNumber=…BirthDate=…
PK = EmployeeName
RK = CertName
PK = CertName
RK = EmployeeName
How do you Partition the data?• EmployeeName or CertName?
![Page 25: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/25.jpg)
Azure Tables Partitions
Employees (Table) Certifications (Table)
(PK) EmployeeName(E1), (RK) CertName (C1)(PK) EmployeeName(E1), (RK) CertName
(C2)
(PK) EmployeeName(E2), (RK) CertName
(C1)
(PK) EmployeeName(E2), (RK) CertName
(C3)
(PK) EmployeeName(E3), (RK) CertName
(C2)
(PK) EmployeeName(E3), (RK) CertName
(C3)
How do you Partition the data?• EmployeeName or CertName?
(PK) CertName (C1), (RK) EmployeeName(E1)
(PK) CertName (C1), (RK)
EmployeeName(E2)
(PK) CertName (C2), (RK)
EmployeeName(E1)
(PK) CertName (C2), (RK)
EmployeeName(E3)
(PK) CertName (C3), (RK)
EmployeeName(E2)
(PK) CertName (C3), (RK)
EmployeeName(E3)
O
R
![Page 26: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/26.jpg)
Azure Tables
EmployeeName BirthDate FavoriteTeam
David Anderson 1/1/1970
Nancy Wilson 4/15/1965 Atlanta Falcons
John Doe April 1, 1989
No Fixed Schema for
entities
Employee table
![Page 27: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/27.jpg)
Azure Table
Demo
![Page 28: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/28.jpg)
Azure Tables
Summary
● No nice relational schema model that we get in RDBMS
● Easy to work with
● Built to scale
● Great pricing
![Page 29: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/29.jpg)
DocumentDB• Azure Document DB - relatively new service
• It’s fully managed Document database as a service
• It stores the data in JSON
• The Azure Tables are extremely scalable and cheap, but if you start querying on any other attributes then you start running into problems.
• It’s massively scalable but fully query-able by all parts of JSON tree.
• Capacity units are 10GB in size and predictable reads and writes / second.
![Page 30: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/30.jpg)
DocumentDB
![Page 31: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/31.jpg)
DocumentDB
Demo
![Page 32: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/32.jpg)
Azure Service Bus
● Secure messaging capabilities
● Enable loosely coupled solutions
● Achieve Publish-Subscribe scenarios
![Page 33: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/33.jpg)
Tightly Coupled
Store Front
End
Store Back
End
Order
processing
Shipping
![Page 34: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/34.jpg)
Loosely Coupled
Store Front
End
Store Back
End
Order
processing
Shipping
Order Queue
![Page 35: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/35.jpg)
Loosely Coupled
Store Front
End
Store Back
End
Order
processingOrder Queue
Shipping
![Page 36: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/36.jpg)
Service Bus – Queue
Demo
![Page 37: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/37.jpg)
Publish - Subscribe
Store Front
End
Store Back
End
Order
processingOrder Queue
Order Printers
Subscription
1
Subscription
2
Shipping
Order
Papers
![Page 38: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/38.jpg)
Service Bus – AMQP• Service Bus supports the Advanced Message Queueing Protocol (AMQP) 1.0.
• AMQP enables you to build cross-platform, hybrid applications using an open standard protocol
• Reliable: The AMQP 1.0 protocol allows messages to be exchanged with a range of reliability guarantees, from fire-and-forget to reliable, exactly-once acknowledged delivery.
• Using AMQP from .Net:Endpoint=sb://[namespace].servicebus.windows.net;SharedSecretIssuer=[issuer
name];SharedSecretValue=[issuer key];TransportType=Amqp
• Using AMQP from Java:# servicebus.properties - sample JNDI configuration# Register a ConnectionFactory in JNDI using the form:# connectionfactory.[jndi_name] = [ConnectionURL]connectionfactory.SBCF = amqps://[username]:[password]@[namespace].servicebus.windows.net
![Page 39: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/39.jpg)
Service Bus – AMQP
![Page 40: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/40.jpg)
Cloud PatternsResiliency Patterns
● Circuit Breaker PatternHandle faults that may take a variable amount of time to rectify when connecting to a remote service or resource. This pattern can improve the stability and resiliency of an application.
● Retry PatternEnable an application to handle temporary failures when connecting to a service or
network resource by transparently retrying the operation in the expectation that the failure is transient. This pattern can improve the stability of the application.
● Compensating Transaction PatternUndo the work performed by a series of steps, which together define an eventually consistent operation, if one or more of the operations fails. Operations that follow the eventual consistency model are commonly found in cloud-hosted applications that implement complex business processes and workflows.
![Page 41: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/41.jpg)
SummaryWe Looked at…
- Azure as Infrastructure (VMs)
- Azure as Platform -Websites- Storage - Azure Tables, DocumentDB- Messaging – Service Bus, Queues, Topics, AMQP- Azure CDN
![Page 42: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/42.jpg)
Summary
Network load balancer
VM1 VM2 VM3 VM4
SQL DatabaseAzure
TablesDocumentDB
Service Bus Queue
Subscription 1 Subscription 2
Worker Role 1 Worker Role 2
Cach
e
![Page 43: Cloud architectural patters and Microsoft Azure tools](https://reader034.vdocuments.net/reader034/viewer/2022052601/5593b0621a28ab9c078b4643/html5/thumbnails/43.jpg)
References
- MSDN (cloud architectural patterns)
- azure.microsoft.com (Learning videos and documentation)
- some images from www