techniques for scaling application with security and visibility in cloud
TRANSCRIPT
Techniques for Scaling Application
with Security and Visibility in Cloud
Akshay Mathur@akshaymathu of @appcito
Let’s Know Each Other
• Do you Manage applications?• Hosting providers?• Priorities?• Tools?• Why are you attending?
• What are your Goal?• Happy Users, Happy DevOps, Happy Servers
2@akshaymathu
Akshay Mathur
• 15+ years in IT industry• Currently Product Manager at Appcito• Mostly worked with Startups
• From Conceptualization to Stabilization• At different functions i.e. development, testing, release, marketing, devops• With multiple technologies
• Founding Team Member of• ShopSocially (Enabling “social” for retailers)• AirTight Neworks (Global leader of WIPS)
@akshaymathu 3
4
Ground Rules
• Tweet now: #TechNext @akshaymathu @appcito• Disturb Everyone later
• Not by phone rings• Not by local talks• By more information and questions
@akshaymathu
How Applications are Changing
@akshaymathu 6
Traditional Application
• Monolithic components• All application layers in a box• Complex objects
• Box specific sessions• Designed for vertical scale• Self managed deployment
@akshaymathu 7
New Age Scalability
@akshaymathu 8
Cloud Computing Landscape
@akshaymathu 9
Architectural Mind-shift
@akshaymathu 10
Modern Application
• Light weight services• Application layers designed for
network communication
• Cloud deployment• Designed for horizontal scale
@akshaymathu 11
Growing Applications
@akshaymathu 13
Growth Phase 1: Load Balancing
• Replicate the box• Have a load balancer
@akshaymathu 14
Questions before Growing Further• About Insights:
• Are all server instances healthy?• When should I add more servers?• What is the traffic volume and its
pattern?• What areas of application are used
most?• What are problematic areas?• Who access my application? • What devices, browsers, apps are in
use?
• About Optimization:• How can I serve more traffic using
existing servers?• Does all the serves must be of same
type, running same code?• Can the content be compressed,
cached?• What to do for optimizing content for
various devices?• Do I really need to redirect traffic to a
different URLs for specific servings?• Does managing so many URLs for same
functionality makes sense?• Can someone take care of SSL
termination?
@akshaymathu 15
Growth Phase 2: Insights
• Google Analytics, Statcounter etc. only provide information after page load
• Information about programmatic access is missing
• Access logs provide true information about traffic• Logs are typically in each box rather than a central
place• Difficult to read; log parsers also provide minimal
information
• Need to push logs to some analytics engine and configure analytics engine for getting meaningful information out
@akshaymathu 16
Growth Phase 3: Content Optimization
• Compressing the response• Optimizing images• In-lining the external resources
• JS• CSS• Images as base64
• Caching (as needed)• Prefetching (if possible)
• Google’s PageSpeed does it well for HTML pages
@akshaymathu 17
Growth Phase 4: Offloading
• SSL Handshake• Encryption and Decryption• Connection handling• Content optimization• Anything that can be done asynchronously
e.g. sending email, tweets etc.
• Point solutions are available for each of these
App ServersApache + Pylons
Message QueueRabbitMQ
Background Worker Nodes
Celery
SSL Terminator
Content Optimizer
@akshaymathu 18
Growth Phase 5: Content Switching
• Serve different content from different servers (reverse proxy)• Static files (JS, CSS, Images) may be served from a web server; App server is not
needed• High frequency requests may be served from different server• Different app servers may be used for the use case they are optimized for
• Have different set of servers for different geographies• Dedicate a few servers for specific customer• Dedicate servers for specific functions e.g. authentication, API serving etc.
• HA Proxy is most popular tool here• NginX is also used as reverse proxy
Web ServersNginX
App ServersMongral + Brubeck
App ServersApache + Pylons
Web ServersApache + Wordpress
NoSql DatastoreRedis
NoSql DatastoreMongoDB
Sql DatastoreMySql
Corporate website
Main dynamic content
High frequency requests
High speed storage Main Storage
Content Switching
Reverse Proxy
@akshaymathu 20
Growth Phase 6: Denying BOT Traffic
• Traffic from bad BOTs is about 30%• Amounts to 30% wastage of server
resources
• Various fingerprinting techniques are there for identifying the BOTs
• IP reputation• UA analysis• Pattern analysis• JS insertion• Advance algorithms
@akshaymathu 21
Growth Phase 7: Preventing Data Theft• Typical ways are:
• SQL/object injection• Cross Site Scripting (XSS)• File include• Malware inclusion• Exploiting vulnerabilities of coding, framework,
language, platform
• Scan the deployment regularly• Fix any vulnerability by applying patches• Use Web Application Firewall (WAF)
@akshaymathu 22
Growth Phase 8: Preventing from DDoS Attack
• Volumetric attack• Many clients make connections with
server• Clients send huge traffic to the server• Traffic is typically bogus
• Prevention• Rapidly increase scale to consume
connections/traffic• Rate limit connections/requests• Delay/Deny bogus traffic• Blacklist BAD clients
• Protocol exploits• Attacker crafts traffic knowing the
timeouts and limits of protocol• Slow moving bogus traffic hogs
resources of server
• Prevention• Setup policy to apply aggressive limits
and timeouts in case of heavy load• Terminate connection when unusual
behavior is observed• Blacklist BAD client
@akshaymathu 23
Growth Phase 7: Continuous Delivery• Upgrade the system without disturbing availability• Why Continuous Delivery?
Continuous Delivery• Considerations:
• Zero down time• Even a little downtime means a lot for
high volume applications
• Seamless re-orientation of live traffic from old to new deployment
• User experience has to be smooth
• Easy roll back• Minimize the impact in case something
goes wrong
• Technique: Blue Green deployments
• Deploy old and new version in parallel and switch the traffic
• Switch using DNS• Switch using fixed NATed IP addresses• Switch using external tools like load
balancer or reverse proxy
25@akshaymathu
App & Traffic Metrics
What is Needed Overall?
26
Availability Performance Security DevOps
Advanced Load Balancing
Content Switching
Application Fluency
Elastic & Self-Scaling
Continuous Deployment
Request Mirroring
Request Replay
Programmable Policies
Per Application Control
Front-End Optimization
Mobile and Web Client App
optimization
Caching & compression
Predictive API caching
Application & Server offloading
Application Firewall
Elastic SSL
Anomaly Detection
DDoS Prevention
BOT Protection
Trends & Correlations Anomalies Policy
Recommendations
Analytics & Insights
@akshaymathu 27
CDN
Custom Scripts, Rules, Alert Management Aggregation across instances
Application Front-End Architecture
• Spaghetti of point solutions• Multiple points of failure, redundancy difficult to setup• Not elastic and cloud native
@akshaymathu 28
CDN
Application Front-End Architecture with CAFE
• All services for application under one consolidated product• Easy Activation of capabilities closer to application• Application policy is coordinated across services and policy enforced
Availability
Security Performance Continuous Deployment
Appcito Cloud Application Front-End (CAFE)
Cloud Application Front End (CAFE)
Taking Cloud Applications from Good to Great
Appcito CAFE Service
Insights & Analytics
Content Optimization
Application Security & DDoS
Prevention
Unified Functionality Available As
SaaS Delivery
Simple Activation
No Code Change
For
Dev /OpsCloud-agnostic
App Owner
ElasticContinuous
Delivery
Availability & Elasticity
Typical Deployment
Customer’s Cloud
Customer’sEnd Users
app server
app server
Load Balancer
app server
DNS
Network Subnet
Availability Zone
Deployment with CAFECustomer’s Cloud
Customer’sEnd Users
app server
app server
Load Balancer
app server
Appcito Cloud
CAFE Barista
Management, Control, Analytics
DNS
CAFEPEP
Network Subnet
Availability Zone
@akshaymathu 33
CAFE Configuration Model• Think Out of the box (literally)• Think in terms of
• Applications• Traffic flow• Request patterns
• Forget about• Box provisioning• Box configuration• Networking flow• L2/L3 access control
Production A (Blue)
Production B (Green)
Launch
Upgrade
Traffic Splitting
80% 20%
Appcito CAFE
80%
20%
CAFE Blue/Green Technique
• Steer traffic NOT switch• Test with production traffic• Move with confidence
• Compare performance and take informed decisions
App & Traffic Metrics
Appcito CAFE Service Capabilities
35
Availability Performance Security DevOps
Advanced Load Balancing
Content Switching
Application Fluency
Elastic & Self-Scaling
Continuous Deployment
Request Mirroring
Request Replay
Programmable Policies
Per Application Control
Front-End Optimization
Optimization for client
Caching & compression
Predictive caching
Application & Server offloading
Application Firewall
Elastic SSL
Anomaly Detection
DDoS
BOT Protection
Trends & Correlations
Anomalies Detection
Policy Recommendation
Analytics & Insights
