build highly scalable_low_latency_applications

Building Highly Scalable and High Scalable Applications using Java/JEE Technologies

Shivnarayan VarmaSr. Architect

Tech COE, Fiserv

© 2010 Fiserv, Inc. or its affiliates.

Agenda :

• Defining and understanding scalability.• Scalability variables & Factors• Building our own Scalable Application• (Scale seamlessly

• High Performance• Unlimited storage expansion• Reduce Network Latency

• Direct the user to nearest server• Scale out, no Scale in• Sweet spot: Low Cost, High Performance, Scalable)• Scalability Real Time Issue

© 2012 Fiserv, Inc. or its affiliates. •3

• How do I know if I have a scalability problem? • Why we need scalability?

Defining and understanding scalability.

© 2012 Fiserv, Inc. or its affiliates. 4

Scalability Variables

• Scalability - Number of users / sessions / transactions / operations the entire system can perform

• Performance – Optimal utilization of resources• Responsiveness – Time taken per operation• Availability - Probability of the application or a portion of the

application being available at any given point in time• Downtime Impact - The impact of a downtime of a

server/service/resource - number of users, type of impact etc• Cost• Maintenance Effort

© 2012 Fiserv, Inc. or its affiliates. 5

Factors for Building Robust Scalable JEE Application

• Platform selection• Hardware• Network limits• Garbage Collection• Application Design/ Architecture• Database Structure and Architecture• Deployment Architecture• Storage Architecture• Monitoring mechanisms

Typical JEE Reference Architecture

7

Building our own Scalable Application.

Platform selectionHardwareApplication DesignDatabase/Data store Structure and ArchitectureDeployment ArchitectureStorage ArchitectureMonitoring Mechanism

© 2012 Fiserv, Inc. or its affiliates. 8

Identify areas of bottlenecks

• Database red/Write• Memory Clustering• Failover Caching

© 2012 Fiserv, Inc. or its affiliates. •9

Typical JEE Reference Architecture

© 2012 Fiserv, Inc. or its affiliates.

© 2012 Fiserv, Inc. or its affiliates. 11

JEE Scalability Points

• We will now build an example architecture for an example app using the following iterative incremental steps –• Inspect current Architecture• Identify Scalability Bottlenecks• Identify SPOFs and Availability Issues• Identify Downtime Impact Risk Zones• Apply one of -

• Vertical Scaling• Vertical Partitioning• Horizontal Scaling• Horizontal Partitioning

• Repeat process

© 2012 Fiserv, Inc. or its affiliates. 12

JEE Scalability Points

• We will now build an example architecture for an example app using the following iterative incremental steps –• Inspect current Architecture• Identify Scalability Bottlenecks• Identify SPOFs and Availability Issues• Identify Downtime Impact Risk Zones• Apply one of -

• Vertical Scaling• Vertical Partitioning• Horizontal Scaling• Horizontal Partitioning

• Repeat process

© 2012 Fiserv, Inc. or its affiliates. 13

Clustering factors

• High Availability• Load balancing• Fault Tolerance• Failover• Idempotent methods

© 2012 Fiserv, Inc. or its affiliates. 14

Best Practices

• Use scalable session replication mechanismsUse collocated deployment instead of distributed one Shared resources and servicesDistributed cache

• Memcached/Terracotta• Paralleled processing• MapReduce• MPI• Using unorthodox approach to achieve high scalability

© 2012 Fiserv, Inc. or its affiliates.

How can I integrate multiple applications so that they work together and can exchange information?

Remote Procedure Invocation

File Transfer Shared Database

Messaging- MOM

© 2012 Fiserv, Inc. or its affiliates.

Asynchronous MOM Element

Channels are separate from applications

Channels are asynchronous & reliable

© 2012 Fiserv, Inc. or its affiliates.

How does one application communicate with another using messaging?

Messaging Channel -A virtual pipe that connects a sender to a receiver application.

© 2012 Fiserv, Inc. or its affiliates.

Point To Point Channel

How can the sender be sure that exactly one receiver will receive the message?

In point-to-point Java Message Service (JMS) , messages are routed to an individual consumer which maintains a queue of "incoming" messages. This messaging type is built on the concept of message queues, senders, and receivers

© 2012 Fiserv, Inc. or its affiliates.

Publish – Subscribe Channel

How can the sender broadcast a message to all interested receivers?

In JMS publish – subscribe ,subscribers may register interest in receiving messages on a particular message topic. In this model, neither the publisher nor the subscriber knows about each other.

20

Message Routing

© 2012 Fiserv, Inc. or its affiliates. 21

Web Banking Page

• Partitioning•HTTP Caching•RDBMS Sharding•NOSQL• Distributed Caching•Data Grids•ConcurrencyScalability Patterns: State

© 2012 Fiserv, Inc. or its affiliates. 22

Advance Confirmation Page

• HTTP Caching• Reverse Proxy• • Varnish• • Squid• • rack-cache• • Pound• • Nginx• • Apache mod_proxy• • Traffic Server

© 2012 Fiserv, Inc. or its affiliates. 23

Successful Creation of RA

• Generate Static Content• Precompute content• • Homegrown + cron or Quartz• • Spring Batch• • Gearman• • Hadoop• • Google Data Protocol• • Amazon Elastic MapReduce

© 2012 Fiserv, Inc. or its affiliates. 24

© 2012 Fiserv, Inc. or its affiliates. 25

HTTP CachingSubsequent request

© 2012 Fiserv, Inc. or its affiliates. 26

How to scale out RDBMS?

• Sharding• •Partitioning• •Replication

© 2012 Fiserv, Inc. or its affiliates. 27

Sharding: Partitioning

© 2012 Fiserv, Inc. or its affiliates. 28

NOSQL in the wild

•Google: Bigtable• Amazon: Dynamo• Amazon: SimpleDB• Yahoo: HBase• Facebook: Cassandra• LinkedIn: VoldemortNOSQL in the wild

© 2012 Fiserv, Inc. or its affiliates. 29

Sharding: Partitioning

• Relational DBs (MySQL, Oracle, Postgres)• Object DBs (Gemstone, db4o)• Clustering products (Coherence,Terracotta)• Most caching products (ehcache)

© 2012 Fiserv, Inc. or its affiliates. 30

Sharding: Partitioning

• Types of NOSQL stores• • Key-Value databases (Voldemort, Dynomite)• • Column databases (Cassandra, Vertica, Sybase IQ)• • Document databases (MongoDB, CouchDB)• • Graph databases (Neo4J, AllegroGraph)• • Datastructure databases (Redis, Hazelcast)

© 2012 Fiserv, Inc. or its affiliates. 31

Distributed Caching

• •Write-through• •Write-behind• • Eviction Policies• • Replication• • Peer-To-Peer (P2P)• Distributed Caching

© 2012 Fiserv, Inc. or its affiliates. 32

Distributed Caching

• Pain of Duplication• • How do I get failover capability while avoiding• excessive duplication of data?

© 2012 Fiserv, Inc. or its affiliates. 33

Distributed Caching

© 2012 Fiserv, Inc. or its affiliates. 34

References

• http://www.theserverside.com/news/1320914/Scaling-Your-Java-EE-Applications-Part-2