7 secrets of wells fargo soa platform's high availability
DESCRIPTION
7 Secrets of Wells Fargo SOA platform's high availabilityTRANSCRIPT
![Page 1: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/1.jpg)
Ram Lakshmanan
Sr. Systems Architect
Java One – 9/23/2013
secrets of Wells Fargo SOA
platform's 99.99% availability
![Page 2: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/2.jpg)
About Wells Fargo
Just two data points
– Serving 1 in 3 US households
– Originating 1 in 3 US mortgages
![Page 3: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/3.jpg)
About Wells Fargo
Just two data points
– Serving 1 in 3 US households
– Originating 1 in 3 US mortgages
Global SIFI (Systemically Important Financial Institution)
![Page 4: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/4.jpg)
About Wells Fargo
Just two data points
– Serving 1 in 3 US households
– Originating 1 in 3 US mortgages
Global SIFI (Systemically Important Financial Institution)
4th in assets & 1st in market value among US peers
No. 1 most valuable financial services brand in the world (Brand Finance, The Banker)
![Page 5: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/5.jpg)
Agenda
High Level Architecture
Secrets
1. AAA…
2. One 9
3. Lord protect me from my friends
4. 240k
5. Cockpit
6. Wackamole
7. *
Meta-Secret
![Page 6: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/6.jpg)
High Level Architecture
![Page 7: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/7.jpg)
High Level Architecture
Online App
Mobile App
Stores App
Call Center App
Other LOB Apps (100+)
![Page 8: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/8.jpg)
High Level Architecture
SOR-2
SOR-60
SOR-5
SOR-3
SOR-4
SOR-6
SOR-1
Online App
Mobile App
Stores App
Call Center App
Other LOB Apps (100+)
![Page 9: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/9.jpg)
High Level Architecture
SOR-2
SOR-60
SOR-5
SOR-3
SOR-4
SOR-6
SOR-1
Online App
Mobile App
Stores App
Call Center App
Other LOB Apps (100+)
SOA <SOAP/>
HTTP
JDBC
JMS
HTTPS
LDAP
![Page 10: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/10.jpg)
Agenda
High Level Architecture
Secrets
1. AAA…
2. One 9
3. Lord protect me from my friends
4. Cockpit
5. 240k
6. Wackamole
7. *
Meta-Secret
![Page 11: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/11.jpg)
Secret 1: AAA…
datacenters: active-active-active…
![Page 12: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/12.jpg)
Secret 2: One 9
![Page 13: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/13.jpg)
Secret 2: One 9
With “intelligent” Software that is running on one 9 (i.e. 90%) hardware availability you can achieve Four 9s (i.e. 99.99%) availability
SOR SOA
SOR SOA
Data Center 1
Data Center 2
![Page 14: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/14.jpg)
Secret 2: One 9
With “intelligent” Software that is running on one 9 (i.e. 90%) hardware availability you can achieve Four 9s (i.e. 99.99%) availability
SOR SOA
SOR SOA
Data Center 1
Data Center 2
![Page 15: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/15.jpg)
Secret 2: One 9
With “intelligent” Software that is running on one 9 (i.e. 90%) hardware availability you can achieve Four 9s (i.e. 99.99%) availability
SOR SOA
SOR SOA
Data Center 1
Data Center 2
![Page 16: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/16.jpg)
Secret 2: One 9
With “intelligent” Software that is running on one 9 (i.e. 90%) hardware availability you can achieve Four 9s (i.e. 99.99%) availability
SOA JMS Server
![Page 17: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/17.jpg)
Secret 2: One 9
With “intelligent” Software that is running on one 9 (i.e. 90%) hardware availability you can achieve Four 9s (i.e. 99.99%) availability
SOA JMS Server
storage
![Page 18: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/18.jpg)
Secret 3: Lord protect me from my friends
SOR 2
SOR 60
SOR 3
SOR 1
SOA
![Page 19: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/19.jpg)
Secret 3: Lord protect me from my friends
SOR 2
SOR 60
SOR 3
SOR 1
SOA
![Page 20: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/20.jpg)
Secret 3: Lord protect me from my friends
SOR 2
SOR 60
SOR 3
SOR 1
SOA
SOR 2 SOA
![Page 21: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/21.jpg)
Secret 3: Lord protect me from my friends
SOR 2
SOR 60
SOR 3
SOR 1
SOA
SOR 2 SOA
![Page 22: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/22.jpg)
Secret 3: Lord protect me from my friends
SOR 2
SOR 60
SOR 3
SOR 1
SOA
SOR 2 SOA
![Page 23: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/23.jpg)
Secret 3: Lord protect me from my friends
SOR 2
SOR 60
SOR 3
SOR 1
SOA
SOR 2 SOA
![Page 24: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/24.jpg)
Secret 3: Lord protect me from my friends
SOR 2
SOR 60
SOR 3
SOR 1
SOA
SOR 2 SOA
![Page 25: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/25.jpg)
Secret 3: Lord protect me from my friends
SOR 2
SOR 60
SOR 3
SOR 1
SOA
SOR 2 SOA
SOR 2
SOA
![Page 26: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/26.jpg)
Secret 3: Lord protect me from my friends
Fault Containment
SOR 2
SOR 60
SOR 3
SOR 1
SOA
SOR 2 SOA
SOR 2
SOA
![Page 27: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/27.jpg)
Secret 3: Lord protect me from my friends
Fault Containment
Fail Fast (Exponential Backoff)
SOR 2
SOR 60
SOR 3
SOR 1
SOA
SOR 2 SOA
SOR 2
SOA
![Page 28: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/28.jpg)
Secret 4: 240k
![Page 29: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/29.jpg)
Secret 4: 240k
80K regression tests
![Page 30: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/30.jpg)
Secret 4: 240k
80K regression tests
3x production load testing
![Page 31: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/31.jpg)
Secret 4: 240k
80K regression tests
3x production load testing
EDS – Enterprise Data Simulator
![Page 32: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/32.jpg)
Secret 5: Cockpit
Pilot*
* - if one thing you want to take away from this session
![Page 33: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/33.jpg)
Secret 5: Cockpit
Pilot*
Switches (installing without outage)
* - if one thing you want to take away from this session
![Page 34: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/34.jpg)
Secret 6: Wackamole
http://www.youtube.com/watch?v=6apVZWu_1LU
![Page 35: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/35.jpg)
Secret 6: Wackamole
Out of control – CPU will spike, memory will leak, JVM will crash.
sometimes
^
http://www.youtube.com/watch?v=6apVZWu_1LU
![Page 36: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/36.jpg)
Secret 6: Wackamole
Out of control – CPU will spike, memory will leak, JVM will crash.
Strong monitoring & instant reactions
sometimes
^
http://www.youtube.com/watch?v=6apVZWu_1LU
![Page 37: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/37.jpg)
Secret 7: .*
![Page 38: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/38.jpg)
Secret 7: .*
Dangerous species
![Page 39: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/39.jpg)
Secret 7: .*
Dangerous species
Restart Button
![Page 40: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/40.jpg)
Secret 7: .*
Dangerous species
Restart Button Factory Reset
![Page 41: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/41.jpg)
Secret 7: .*
Dangerous species
Restart Button
SODA – Service Oriented Development & Assembly
Factory Reset
![Page 42: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/42.jpg)
Secret 7: .*
Dangerous species
Restart Button
SODA – Service Oriented Development & Assembly
Not ‘So Cool’
– Never chosen technology for the sake of “Coolness”
– But full stack is running almost on Open Source technologies
Factory Reset
![Page 43: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/43.jpg)
Agenda
High Level Architecture
Secrets
1. AAA…
2. One 9
3. Lord protect me from my friends
4. 240k
5. Cockpit
6. Wackamole
7. *
Meta-Secret
![Page 44: 7 Secrets of Wells Fargo SOA platform's high availability](https://reader033.vdocuments.net/reader033/viewer/2022051818/54c350cd4a7959b4208b46c6/html5/thumbnails/44.jpg)
Meta-Secret: Our Team