scaling mature systems

Scaling Mature Systems

© OPENJAW TECHNOLOGIES 20122

Teaching an

old dog new

tricks

© OPENJAW TECHNOLOGIES 20123

Change in environment

• Different usage or purpose

• Increase in products

• Increase in traffic

© OPENJAW TECHNOLOGIES 20124

Morten JørgensenChief Architect

OpenJaw Technologies

Focus:

• Support product functionality

• Operational efficiency

• Scalability and fault tolerance

© OPENJAW TECHNOLOGIES 20125

System Load

Reso

urc

e C

on

su

mp

tio

n

Scalability vs Performance

© OPENJAW TECHNOLOGIES 20126

• Distributed transactions

• Resource contention

• State/session management

• Deployment process

Common Scalability Blockers

Distributed Transactions

© OPENJAW TECHNOLOGIES 20128

• ACID principle

• Pessimistic

• Slow

• BASE principle

• Optimistic

• Fast

ACID vs. BASE

© OPENJAW TECHNOLOGIES 20129

• Come at a cost:

• Latency

• Performance

• Scalability

• Availability

• Software “failure” may make business sense!

Distributed Transactions

© OPENJAW TECHNOLOGIES 201210

Distributed Transactions

© OPENJAW TECHNOLOGIES 201211

• Reconfigure transaction management

• Implement failure detection

• Report or log failures

• Implement failure handling processes

• Convince business owner

Distributed Transactions

Resource Contention

© OPENJAW TECHNOLOGIES 201213

• Contention around internal and external systems

• Consumer experience limited by availability,

performance and scalability of these systems

• Break dependency by using asynchronous

processes

• Caching is the easiest form of asynchronous

decoupling

Resource Contention

© OPENJAW TECHNOLOGIES 201214

• Implement using SQL or no-SQL database

• Choose platform based on your needs

• Simple approach

• Build cache key

• Cache lookup

• Cache storage

Cache Implementation

© OPENJAW TECHNOLOGIES 201215

• Trade-off between data quality and performance

• Measure cache hit-rate

• Measure data quality

• Tune cache according to both!

Cache Tuning

State & Session Management

© OPENJAW TECHNOLOGIES 201217

• Make your application stateless

• Remove session data from the application

• Increase in user sessions will not impact

single node

• Add new nodes when traffic increases

Stateless Applications

© OPENJAW TECHNOLOGIES 201218

• Two options:

• Move to new platform

• Change existing platform

• J2EE conformance allowed us to use Tomcat

• Tomcat allowed session manager plugin

Stateless Applications

© OPENJAW TECHNOLOGIES 201219

Cassandra

KeySpace

Column Family

Key

Column

Schema

Table

Row

Column

SuperColumn

Column

Key

Session Manager

Session ID

Session Attribute Name

Session Object Value

© OPENJAW TECHNOLOGIES 201220

Cassandra Session Manager

Cassandra“Ring”

Sta

tele

ss L

oad

Bala

ncer

Thrift

Application

Tomcat PLUGIN

Application

Tomcat PLUGIN

Application

Tomcat PLUGIN

© OPENJAW TECHNOLOGIES 201221

• Session object serialisation

• Session object synchronisation

• Network bandwidth

Cassandra Session Manager

Deployment Process

© OPENJAW TECHNOLOGIES 201223

• Your application is decoupled

• Your application is stateless

• You can add nodes when demand increases

• How quickly can you bring a new node up?

• Is your application an elastic resource?

Deployment Process

© OPENJAW TECHNOLOGIES 201224

• Remove everything but code!

• Extract session state

• Extract configuration

• Create a VM image as a template

• Instantiate VMs from the image on demand

Deployment Process

© OPENJAW TECHNOLOGIES 201225

Deployment Process

VM VM VMVM

Stateless Load Balancer

Client

AdminConsole

Results

© OPENJAW TECHNOLOGIES 201227

• Product built over 10 years

• Large solution with high complexity

• Made Internet scalable with small amount of

engineering resources

• Ability to scale is now a key selling point

Our Results

Thank you!