(pfc304) effective interprocess communications in the cloud: the pros and cons of microservices...
DESCRIPTION
You can't (yet) bend the law of Physics, but you can use the power of the cloud to design applications that run as fast as the speed of light! This session will focus on the best practices for optimizing performance to the very last millisecond. We'll dive into topics such as caching at every layer of your application, TCP optimizations, SSL optimizations, latency based routing, and much more. These best practices can help you to streamline your infrastructure utilization, improve performance and allow you to scale economically.TRANSCRIPT
• Netflix – background and evolution
• Netflix – background and evolution
• Monolithic Apps
Load Balancer
Monolithic App
Account
ComponentCatalog
Component
Recommendation
Component
Customer Service
Component
Database
Time
User Accounts
Shopping Cart
Product Catalog
Customer Service
Product Catalog Customer Service
Tipping Point
& &
Organizational Growth Disverse Functionality Bottleneck in
Monolithic stack
MicroServices
Perhaps?
• Netflix – background and evolution
• Monolithic Apps
• Microservices
MicroService
MegaService
Attribution: Adrian Cockroft, Martin Fowler …
• Write programs that do one thing and do it well.
• Write programs to work together.
tr 'A-Z' 'a-z' < doc.txt | tr -cs 'a-z' '\n' | sort | uniq | comm -23 -
/usr/share/dict/words
Program to print misspelt words in doc.txt
Comparing Monolithic to MicroServices
Load Balancer
Monolithic App
Account
ComponentCatalog
Component
Recommendation
Component
Customer Service
Component
Database
Load Balancer
Account
ServiceCatalog
Service
Recommendation
ServiceCustomer Service
Service
Catalog
DB
API Gateway
Customer
DB
• Netflix – background and evolution
• Monolithic Apps
• Microservices
– Why?
Your
App/Service
Service X
Service Y
Service Z
Service L
Service M
• Netflix – background and evolution
• Monolithic Apps
• Microservices
– Challenges and Solutions
Vs
Heterogeneity
Can lead to chaos if not designed right …
non-JVM
Service Discovery
What’s your address?
Account
ServiceCatalog
Service
Recommendation
ServiceCustomer Service
Service
X
Service Y
Service
Z
Service
Registry
Service
(e.g. Netflix Eureka)
2 Billion
~20 Billion Fan
1 Request 1 Request
Monolithic
AppMicroServices
Your
App/Service
Service X
Service Y
Service Z
Cache
ClusterCache
Cluster
Tip: Config your TTL based on flexibility with data staleness!
Your
App/Service
Service X
Service Y
Service Z
Cache
Cluster
Fn {A, B, C}
Cache
Cluster
A/B Test
Service
User Account
Service
Service X Service Y Service ZApp
A/B Test
Service
User Account
Service
Service X Service Y Service ZApp
reduces dependency
load
Data Serialization Overhead
Service A Service B Service C Service D
getMovies() getMovie() getMovieMetadata(
)C
l
i
e
n
t
D
C
l
i
e
n
t
C
C
l
i
e
n
t
B
Data
transformation
XAvroX
XmlJSON
Overall Response Time
A single “bad” service can still bring your service down
Your
App/Service
Service X
Service Y
Service Z
Service L
Service M
Your
App/Service
Service X
Service Y
Service Z
Service L
Service M
Your
App/ServiceYour
App/Service
Your
App/Service
Service X
Service Y
Service Z
Service L
Service M
Your
App/ServiceYour
App/Service
Service X
Service Y
Service Z
Service L
Service M
http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/WhatIsAutoScaling.html
http://techblog.netflix.com/2013/11/scryer-
netflixs-predictive-auto-scaling.html
1.
Account
Service 1
Recommendation
Service 1
Customer Service
Service 1
API Gateway
Account
Service N
Recommendation
Service N
Customer Service
Service N
Customer Service
Load Balancer
Reco Service
Load Balancer
Account Service
Load Balancer
ELB
Account
Service 1
Recommendation
Service 1
Customer Service
Service 1
API Gateway
Account
Service N
Recommendation
Service N
Customer Service
Service N
Account Service
LB
Recommendation
Service LB
Customer Service
Service LB
ELB
Service Dependency Visualization
Your Service
XY Farm
How many dependencies does my service have?
What is the Call Volume on my Service?
Are any Dependency Services running Hot?
What are the Top N Slowest “Business Transactions”?
What are the sample HTTP Requests/Responses that had a
500 Error Code in the last 30 minutes?
You
Your Service Dependency
Graph
• Netflix – background and evolution
• Monolithic Apps
• Microservices
– Challenges and Solutions
– Best Practices
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html
three
Fault Injection Testing (FIT)
Device ELB Service B
Service C
Internet Edge
Device or Account Override
Zuul
Service A
Request-level simulations
Refer Re:Invent Talk “Embracing Failure: Fault Injection and
Service Reliability” – PFC305
• Netflix – background and evolution
• Monolithic Apps
• Microservices
– Challenges and Solutions
– Best Practices
• InterProcess Communication
Netflix IPC Stack (1.0)
A
p
a
c
h
e
H
T
T
P
C
l
i
e
n
t
Eureka (Service
Registry)
Server (Karyon)
Apache
Tomcat
Client
H
y
s
t
r
i
x
E
V
C
a
c
h
e
Ribbon
Load
Balancing
Eureka
Integration
Metrics
(Servo)
Bootstrapping (Governator)
Metrics (Servo)
Admin ConsoleHTTP
Eureka Integration
Registration
Fetch Registry
A Blocking Architecture
Netflix IPC Stack (2.0)Client (Ribbon 2.0)
Eureka (Service
Registry)
Server (Karyon)
Ribbon Transport
Load
Balancing
Eureka
Integration
Metrics
(Servo)
Bootstrapping (Governator)
Metrics (Servo)
Admin Console
HTTP
Eureka Integration
Registration
Fetch Registry
Ribbon
Hystrix
EVCache
R
x
N
e
t
t
y
RxNetty
UDP
TCP
WebSockets
SSE
A Completely Reactive Architecture
Details: http://www.meetup.com/Netflix-Open-Source-
Platform/events/184153592/
interface
recommendationsByUserId @Var "userId"
@Http(method = HttpMethod.GET, uriTemplate="/users/{userId}/recommendations")
@Hystrix(fallbackHandler = RecommendationFallbackHandler.class)
@EvCache(name = "movie-rec", appName = "movieService", cacheKeyTemplate = "{userId}")
MovieService movieService = Ribbon.from(MovieService.class);
Observable<Recommendations> result =
movieService.recommendationsByUserId(“user1”)
.toObservable();
NetflixOSS
NetflixOSS
• Eureka
• Karyon
• Ribbon
• Hystrix
• Archaius
• Servo
• EVCache
• Curator/Exhibitor
Talk Time Title
PFC-305 Wednesday, 1:15pm Embracing Failure: Fault Injection and Service Reliability
BDT-403 Wednesday, 2:15pm Next Generation Big Data Platform at Netflix
PFC-306 Wednesday, 3:30pm Performance Tuning EC2
DEV-309 Wednesday, 3:30pm From Asgard to Zuul, How Netflix’s proven Open Source
Tools can accelerate and scale your services
ARC-317 Wednesday, 4:30pm Maintaining a Resilient Front-Door at Massive Scale
PFC-304 Wednesday, 4:30pm Effective Inter-process Communications in the Cloud: The
Pros and Cons of Micro Services Architectures
ENT-209 Wednesday, 4:30pm Cloud Migration, Dev-Ops and Distributed Systems
APP-310 Friday, 9:00am Scheduling using Apache Mesos in the Cloud
• Netflix – background and evolution
• Monolithic Apps
• Microservices
– Challenges and Solutions
– Best Practices
• InterProcess Communication
• Takeaways
http://techblog.netflix.com
http://netflix.github.io
http://martinfowler.com/articles/microservices.html
http://microservices.io/
Please give us your feedback on this session.
Complete session evaluations and earn re:Invent swag.
http://bit.ly/awsevals
Netflix Booth