cloud foundry diego: modular and extensible substructure for microservices
TRANSCRIPT
CLOUD FOUNDRY DIEGOMODULAR AND EXTENSIBLE SUBSTRUCTURE FOR MICROSERVICES
1
MEMatt Stine @mstine
Principal Software EngineerPivotal
2
I WROTE A LITTLE CLOUD BOOK...
FREE - Compliments of Pivotal
http://bit.ly/cloud-native-book
3
MICROSERVICES?4
Loosely coupled service oriented architecture with bounded
contexts...— Adrian Cockcroft
5
YOU MUST BE THIS TALL TO USE
MICROSERVICES▸ RAPID PROVISIONING▸ BASIC MONITORING
▸ RAPID APPLICATION DEPLOYMENT▸ DEVOPS CULTURE
http://martinfowler.com/bliki/MicroservicePrerequisites.html
6
A SYMBIOTIC RELATIONSHIP
7
PLATFORM FEATURES▸ Environment Provisioning▸ On-Demand Scaling▸ Failover/Resilience▸ Routing/Load Balancing
▸ Data Service Operations (BOSH)▸ Monitoring
8
But we need MOAR features...
9
HARDTO ADD NEW FEATURES
HARDTO MAINTAIN EXISTING FEATURES
10
WHY IS IT HARD?
From https://www.youtube.com/watch?v=1OkmVTFhfLY11
ENTER THEREWRITE
12
WHAT?
From https://www.youtube.com/watch?v=1OkmVTFhfLY13
THE RESULT:
14
THIS IS DIEGO
15
MEET THE RECEPTOR
16
TASKSONE-OFF WORK
LRPSLONG-RUNNING PROCESSES
17
DESIRED TASK{ ...
"rootfs": "docker:///docker-org/docker-image", "env": [ {"name": "ENV_NAME_A", "value": "ENV_VALUE_A"}, {"name": "ENV_NAME_B", "value": "ENV_VALUE_B"} ],
"cpu_weight": 57, "disk_mb": 1024, "memory_mb": 128, "privileged": true,
"action": ACTION(s) TO RUN,
...}
18
ACTIONS▸ RunAction: run proc in container
▸ DownloadAction: fetches and extract archive▸ UploadAction: POST file from container to URL▸ ParallelAction: run multiple actions in parallel▸ SerialAction: runs multiple actions in order
19
MOAR ACTIONS▸ EmitProgressAction: wraps action and logs progress▸ TimeoutAction: wrap action and fail if timed out▸ TryAction: wrap action and ignore errors
20
DESIRED LRP{ ...
"instances": 17,
"rootfs": "VALID-ROOTFS",
"env": [ {"name": "ENV_NAME_A", "value": "ENV_VALUE_A"}, {"name": "ENV_NAME_B", "value": "ENV_VALUE_B"} ],
"cpu_weight": 57, "disk_mb": 1024, "memory_mb": 128, "privileged": true,
"setup": ACTION, "action": ACTION, "monitor": ACTION, "start_timeout": N seconds,
"ports": [8080, 5050],
...}
21
MEET THE BBS
22
LET'S HAVE AN AUCTION
23
LET'S HAVE AN AUCTION
24
LET'S HAVE AN AUCTION
25
INSIDE THE CELL
26
INSIDE THE CELL
27
INSIDE THE CELL
28
INSIDE THE CELL
29
SPECIFICITY GRADIENT
30
REP RESENTING THE ACTUAL WORK
31
WHAT IS THE TRUTH?
32
WHAT IS THE TRUTH?
33
WHAT IS THE TRUTH?
34
CONVERGENCE
35
PLAY WITH DIEGO?
36
DEPLOYING CLOUD FOUNDRY IS HARD
37
https://lattice.cf38
THIS IS LATTICE
39
LATTICE + DIEGO▸ Adds CF GoRouter (dynamic HTTP routing)▸ Adds Doppler (log/metrics aggregation)
▸ LOCAL: vagrant up▸ CLOUD: terraform apply (AWS, Google, DigitalOcean)
40
DEMO41
WHATELSE?
42
MICROSERVICES43
NO MICROSERVICEIS AN ISLAND
44
CHALLENGES OF DISTRIBUTED SYSTEMS▸ Configuration Management
▸ Service Registration & Discovery▸ Routing & Load Balancing
▸ Fault Tolerance (Circuit Breakers!)▸ Monitoring
▸ Concurrent API Aggregation & Transformation45
46
SPRING CLOUDDISTRIBUTED SYSTEM PATTERNS FTW!
47
SPRING CLOUD LATTICE▸ https://github.com/spring-cloud/spring-cloud-
lattice▸ https://spring.io/guides/gs/spring-cloud-and-
lattice
48
EUREKA + RIBBON
49
LATTICE + RIBBON
50
DEMO51
DATAMICROSERVICES?
52
SPRING XD DISTRIBUTED RUNTIME
54
SPRING XD SPI MODEL
55
SPRING XD + LATTICE
56
DEMO57
LEARN MORE!▸ Lattice: http://lattice.cf
▸ Diego Design Notes: https://github.com/cloudfoundry-incubator/diego-design-notes
58
THANKYOU!
59