microsoft: building a massively scalable system with datastax and microsoft's next generation...
TRANSCRIPT
![Page 1: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/1.jpg)
Building a massively scalable system with DataStax and
Microsoft's next generation Paas infrastructure
Rob Bagby Sr. Cloud Architect – Microsoft
www.deveducate.com / www.robbagby.com
@RBagby
Jesus AguilarSr. Cloud Architect – Microsoft
www.giventocode.com
@GivenToCode
![Page 2: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/2.jpg)
How do I scale my Data linearly?Does the VM have the required resources?
Are there any cold VMs that I shouldn't be paying for?
How do I move a service?
Is the VM running?Is the VM too hot?How do I deploy across a part of the cluster?
Does the VM have the required capacity?
How do I roll back a deployment if it is does not work?
How do I deploy across a part of the cluster?
Can I have multiple versions running side-by-side?
Where should I move it?
Does my data have a single point of failure?Is my application running?
Can I replicate data across data centers?
![Page 3: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/3.jpg)
Large and Distributed Challenges
Ops Management High Density
Resource Governance
Deployment and Packaging
Rolling updates
Scalability
Hyper Scale
Load balancing
Scale Out
Maintainability
Granular versioning
In-flight testing
Microservices
Availability
Failure resilient
Rolling updates
Recovery from failures
![Page 4: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/4.jpg)
Azure Service Fabric
VNext PaaS offering
Battle tested – internally used by +5 years in large systems
State of the art orchestration and management
Stateful and Stateless programming model
![Page 5: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/5.jpg)
271 CustomersService Fabric Preview
![Page 6: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/6.jpg)
Azure Service Fabric + Cassandra
Massively scalable system
Cloud powered
Modern architecture patterns
Stateless Services
Cassandra Cluster
![Page 7: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/7.jpg)
Cassandra Deployment Options
Azure Marketplace Templates GitHub Templates
![Page 8: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/8.jpg)
Demo 1 – Cassandra on Azure
![Page 9: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/9.jpg)
Large and Distributed Challenges
Ops Management High Density
Resource Governance
Deployment and Packaging
Rolling updates
Scalability
Hyper Scale
Load balancing
Scale Out
Maintainability
Granular versioning
In-flight testing
Microservices
Availability
Failure resilient
Rolling updates
Recovery from failures
![Page 10: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/10.jpg)
Achieving High Density with Azure Service Fabric
IaaS Approach 1 Cluster – 1 VM
Azure Service Fabric(Stateless, stateful or Actor services)
• 1 service instance per VM with uneven workloads
• Lower compute density
• Slow in deployment & upgrades
• Slower in scaling and disaster recovery
• Many microservices per VM
• High microservices density
• Fast deployment & upgrades
• Fast scaling microservices across the cluster
![Page 11: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/11.jpg)
Placement Constraints
You control your topology
Web
Internal API
N1DMZ = False
N2DMZ = True
N4DMZ = True
N5DMZ = True
N6DMZ = False
N3DMZ = False
Constraints = “(DMZ == False)”
Constraints = “(DMZ == True)”
![Page 12: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/12.jpg)
Demo 2 – The Sample Application
![Page 13: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/13.jpg)
Large and Distributed Challenges
Ops Management High Density
Resource Governance
Deployment and Packaging
Rolling updates
Scalability
Hyper Scale
Load balancing
Scale Out
Maintainability
Granular versioning
In-flight testing
Microservices
Availability
Failure resilient
Rolling updates
Recovery from failures
![Page 14: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/14.jpg)
FD2FD1
UD2
UD1
Nodes will go down.
Fault DomainsSame physical location, HW failures
Upgrade DomainsSW upgrades to the underlying compute resources
Web Web
API API
![Page 15: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/15.jpg)
Rolling UpdatesUD 1
UD 2 UD 3
N1
N2 N3
N4
N5 N6
V1
V1
V1
V2V2V2
![Page 16: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/16.jpg)
Large and Distributed Challenges
Ops Management High Density
Resource Governance
Deployment and Packaging
Rolling updates
Scalability
Hyper Scale
Load balancing
Scale Out
Maintainability
Granular versioning
In-flight testing
Microservices
Availability
Failure resilient
Rolling updates
Recovery from failures
![Page 17: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/17.jpg)
Demo 3 – Handling Failures & Rolling Upgrade
![Page 18: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/18.jpg)
Large and Distributed Challenges
Ops Management High Density
Resource Governance
Deployment and Packaging
Rolling updates
Scalability
Hyper Scale
Load balancing
Scale Out
Maintainability
Granular versioning
In-flight testing
Microservices
Availability
Failure resilient
Rolling updates
Recovery from failures
![Page 19: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/19.jpg)
Scaling and Balancing
• Declarative Model – “I want the topology to look like this”
• Service Fabric does the work• Balances Resources
• Moves services when hot
• Respects constraints
• You provide guidance• These metrics are important
• CPU
• Memory
• Etc.
• Balancing Threshold
• Activity Threshold
![Page 20: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/20.jpg)
Demo 4 – Scalability
![Page 21: Microsoft: Building a Massively Scalable System with DataStax and Microsoft's Next Generation PaaS Infrastructure](https://reader034.vdocuments.net/reader034/viewer/2022042723/587080e81a28ab57368b65c9/html5/thumbnails/21.jpg)
Questions