techniques for scaling application with security and visibility in cloud

Techniques for Scaling Application

with Security and Visibility in Cloud

Akshay Mathur@akshaymathu of @appcito

Let’s Know Each Other

• Do you Manage applications?• Hosting providers?• Priorities?• Tools?• Why are you attending?

• What are your Goal?• Happy Users, Happy DevOps, Happy Servers

2@akshaymathu

Akshay Mathur

• 15+ years in IT industry• Currently Product Manager at Appcito• Mostly worked with Startups

• From Conceptualization to Stabilization• At different functions i.e. development, testing, release, marketing, devops• With multiple technologies

• Founding Team Member of• ShopSocially (Enabling “social” for retailers)• AirTight Neworks (Global leader of WIPS)

@akshaymathu 3

4

Ground Rules

• Tweet now: #TechNext @akshaymathu @appcito• Disturb Everyone later

• Not by phone rings• Not by local talks• By more information and questions

@akshaymathu

How Applications are Changing

@akshaymathu 6

Traditional Application

• Monolithic components• All application layers in a box• Complex objects

• Box specific sessions• Designed for vertical scale• Self managed deployment

@akshaymathu 7

New Age Scalability

@akshaymathu 8

Cloud Computing Landscape

@akshaymathu 9

Architectural Mind-shift

@akshaymathu 10

Modern Application

• Light weight services• Application layers designed for

network communication

• Cloud deployment• Designed for horizontal scale

@akshaymathu 11

Growing Applications

@akshaymathu 13

Growth Phase 1: Load Balancing

• Replicate the box• Have a load balancer

@akshaymathu 14

Questions before Growing Further• About Insights:

• Are all server instances healthy?• When should I add more servers?• What is the traffic volume and its

pattern?• What areas of application are used

most?• What are problematic areas?• Who access my application? • What devices, browsers, apps are in

use?

• About Optimization:• How can I serve more traffic using

existing servers?• Does all the serves must be of same

type, running same code?• Can the content be compressed,

cached?• What to do for optimizing content for

various devices?• Do I really need to redirect traffic to a

different URLs for specific servings?• Does managing so many URLs for same

functionality makes sense?• Can someone take care of SSL

termination?

@akshaymathu 15

Growth Phase 2: Insights

• Google Analytics, Statcounter etc. only provide information after page load

• Information about programmatic access is missing

• Access logs provide true information about traffic• Logs are typically in each box rather than a central

place• Difficult to read; log parsers also provide minimal

information

• Need to push logs to some analytics engine and configure analytics engine for getting meaningful information out

@akshaymathu 16

Growth Phase 3: Content Optimization

• Compressing the response• Optimizing images• In-lining the external resources

• JS• CSS• Images as base64

• Caching (as needed)• Prefetching (if possible)

• Google’s PageSpeed does it well for HTML pages

@akshaymathu 17

Growth Phase 4: Offloading

• SSL Handshake• Encryption and Decryption• Connection handling• Content optimization• Anything that can be done asynchronously

e.g. sending email, tweets etc.

• Point solutions are available for each of these

App ServersApache + Pylons

Message QueueRabbitMQ

Background Worker Nodes

Celery

SSL Terminator

Content Optimizer

@akshaymathu 18

Growth Phase 5: Content Switching

• Serve different content from different servers (reverse proxy)• Static files (JS, CSS, Images) may be served from a web server; App server is not

needed• High frequency requests may be served from different server• Different app servers may be used for the use case they are optimized for

• Have different set of servers for different geographies• Dedicate a few servers for specific customer• Dedicate servers for specific functions e.g. authentication, API serving etc.

• HA Proxy is most popular tool here• NginX is also used as reverse proxy

Web ServersNginX

App ServersMongral + Brubeck

App ServersApache + Pylons

Web ServersApache + Wordpress

NoSql DatastoreRedis

NoSql DatastoreMongoDB

Sql DatastoreMySql

Corporate website

Main dynamic content

High frequency requests

High speed storage Main Storage

Content Switching

Reverse Proxy

@akshaymathu 20

Growth Phase 6: Denying BOT Traffic

• Traffic from bad BOTs is about 30%• Amounts to 30% wastage of server

resources

• Various fingerprinting techniques are there for identifying the BOTs

• IP reputation• UA analysis• Pattern analysis• JS insertion• Advance algorithms

@akshaymathu 21

Growth Phase 7: Preventing Data Theft• Typical ways are:

• SQL/object injection• Cross Site Scripting (XSS)• File include• Malware inclusion• Exploiting vulnerabilities of coding, framework,

language, platform

• Scan the deployment regularly• Fix any vulnerability by applying patches• Use Web Application Firewall (WAF)

@akshaymathu 22

Growth Phase 8: Preventing from DDoS Attack

• Volumetric attack• Many clients make connections with

server• Clients send huge traffic to the server• Traffic is typically bogus

• Prevention• Rapidly increase scale to consume

connections/traffic• Rate limit connections/requests• Delay/Deny bogus traffic• Blacklist BAD clients

• Protocol exploits• Attacker crafts traffic knowing the

timeouts and limits of protocol• Slow moving bogus traffic hogs

resources of server

• Prevention• Setup policy to apply aggressive limits

and timeouts in case of heavy load• Terminate connection when unusual

behavior is observed• Blacklist BAD client

@akshaymathu 23

Growth Phase 7: Continuous Delivery• Upgrade the system without disturbing availability• Why Continuous Delivery?

Continuous Delivery• Considerations:

• Zero down time• Even a little downtime means a lot for

high volume applications

• Seamless re-orientation of live traffic from old to new deployment

• User experience has to be smooth

• Easy roll back• Minimize the impact in case something

goes wrong

• Technique: Blue Green deployments

• Deploy old and new version in parallel and switch the traffic

• Switch using DNS• Switch using fixed NATed IP addresses• Switch using external tools like load

balancer or reverse proxy

25@akshaymathu

App & Traffic Metrics

What is Needed Overall?

26

Availability Performance Security DevOps

Advanced Load Balancing

Content Switching

Application Fluency

Elastic & Self-Scaling

Continuous Deployment

Request Mirroring

Request Replay

Programmable Policies

Per Application Control

Front-End Optimization

Mobile and Web Client App

optimization

Caching & compression

Predictive API caching

Application & Server offloading

Application Firewall

Elastic SSL

Anomaly Detection

DDoS Prevention

BOT Protection

Trends & Correlations Anomalies Policy

Recommendations

Analytics & Insights

@akshaymathu 27

CDN

Custom Scripts, Rules, Alert Management Aggregation across instances

Application Front-End Architecture

• Spaghetti of point solutions• Multiple points of failure, redundancy difficult to setup• Not elastic and cloud native

@akshaymathu 28

CDN

Application Front-End Architecture with CAFE

• All services for application under one consolidated product• Easy Activation of capabilities closer to application• Application policy is coordinated across services and policy enforced

Availability

Security Performance Continuous Deployment

Appcito Cloud Application Front-End (CAFE)

Cloud Application Front End (CAFE)

Taking Cloud Applications from Good to Great

Appcito CAFE Service

Insights & Analytics

Content Optimization

Application Security & DDoS

Prevention

Unified Functionality Available As

SaaS Delivery

Simple Activation

No Code Change

For

Dev /OpsCloud-agnostic

App Owner

ElasticContinuous

Delivery

Availability & Elasticity

Typical Deployment

Customer’s Cloud

Customer’sEnd Users

app server

app server

Load Balancer

app server

DNS

Network Subnet

Availability Zone

Deployment with CAFECustomer’s Cloud

Customer’sEnd Users

app server

app server

Load Balancer

app server

Appcito Cloud

CAFE Barista

Management, Control, Analytics

DNS

CAFEPEP

Network Subnet

Availability Zone

@akshaymathu 33

CAFE Configuration Model• Think Out of the box (literally)• Think in terms of

• Applications• Traffic flow• Request patterns

• Forget about• Box provisioning• Box configuration• Networking flow• L2/L3 access control

Production A (Blue)

Production B (Green)

Launch

Upgrade

Traffic Splitting

80% 20%

Appcito CAFE

80%

20%

CAFE Blue/Green Technique

• Steer traffic NOT switch• Test with production traffic• Move with confidence

• Compare performance and take informed decisions

App & Traffic Metrics

Appcito CAFE Service Capabilities

35

Availability Performance Security DevOps

Advanced Load Balancing

Content Switching

Application Fluency

Elastic & Self-Scaling

Continuous Deployment

Request Mirroring

Request Replay

Programmable Policies

Per Application Control

Front-End Optimization

Optimization for client

Caching & compression

Predictive caching

Application & Server offloading

Application Firewall

Elastic SSL

Anomaly Detection

DDoS

BOT Protection

Trends & Correlations

Anomalies Detection

Policy Recommendation

Analytics & Insights

36

Thanks

@akshaymathu

@akshaymathuakshay@appcito.com