program manager microsoft session...
TRANSCRIPT
What we will cover…Scenarios for a Distributed Cache
What and how Velocity enables
What we will not cover…How to write effective code?
Best practices on ASP.NET or Web Development
Agenda...
Motivation for Distributed Caching
LOB requirements
Driving Forces
Kinds of Cached data
Distributed Cache Usage
Velocity
Overview and Velocity Features
Deployment and Maintenance
Velocity Futures
Conclusions
This section..
LOB requirements
PerformanceConsistent access latency (e.g. 2-3 ms) irrespective of load
(almost) Linear Scale Up – more CPU | N/W | RAM
Out – more machines/server
High AvailabilityServer/machine going down should not result in loss of application availability
Most composite applications designed for serious scale and availability are using mid-tier, distributed caching technologies today.
• Data is sourced from all over the place: local, federated, cloud
• An abundance of applications working together, each with different shapes for the same data
• More workloads require increasing scale, availability, and performance as they become mission critical
• Applications must be cheap to scale
Evolving requirements
• Logic and data distributed, with data living closer to mid-tier logic• Global shared data structures enable
application productivity and acceleration
• Recognition of different types of data with different semantics
• Deployment of high-speed data storage where feature set is not “one size fits all”
Evolving application architecturesbeing solved by
• Rapidly dropping memory prices, enabling huge amounts of data to reside in-memory
• Availability of cheap commodity hardware that can be easily replaced
• Multi-core CPUs, cheap flash memory
Underlying trends
enabled by
Kinds of Data
• A version of the authoritative data
– Aggregated and transformed
• Each version is unique
• Gets refreshed periodically, usually at configured intervals or when authoritativedata changes
• Access pattern: mostly; shared, highly concurrent reads. Local updates for tagging
• Scale: large; replicated
Reference Data (E.g. Product catalog data, Flight Schedules, etc.)
• Data typically generated as part of the application activity
• Gets modified by business transactions
• Retired on completion of a business transaction– Typically logged to a backend data source– Historical data– Could be discarded
• Access pattern: Read and write, exclusive
• Scale: large; partitioned
Activity-oriented Data (E.g. Shopping cart, Purchase order, Session state)
• Domain data typically generated by the application
• Modified by transactions; lives as application data across transactions
• Access pattern: Read and write, concurrent shared read/write
• Scale: large; replicated
Resource-oriented Data (E.g. User Accounts, Auction (Bid/Quote), Seat assignments)
Cache complexity
Low
Medium
High
Caching used to improve service scalability
Caching improves service scalability, facilitates failover
Caching reduces load on authoritative store. Requires Transaction, coherency
Distributed Cache Usage
Commonalities: Extreme transaction volume, store for transient processing
Ho
rizo
nta
lV
ert
ical
s
Scenario
Web
• User-specific HTTP session and shared state across web farm
• In-flight shopping carts for web retail
• Enabling online self-service applications
• Explicit storage of pre-computed or highly-accessed data
LOB• Enterprise-wide product catalog for POS, analytics
• Caching frequently used reference data for a ERP application
Telco• Cellular/VOIP: compute utilization, prepay charges, call routing and session info
• SMS: message content / notification / receipt, billing
Travel • Aggregated flight pricing / availability retrieved from airlines
Defense • Sensor network data processing and threat detection
Financial• Per-user portfolio data and delayed quote storage for trading
• Aggregate and process ticker stream for algorithmic trading
Sample Application
Shopping Cart
Catalog
Inventory
Demo
Distributed Caching
Velocity
What is Velocity?
Where does it fit?
Why Velocity?
When can I Get?
Are customer using it?
Overview and Velocity Features
Deployment and Maintenance
Velocity Futures
Conclusions
This section..
Unified Cache View
What Is "Velocity"?
An explicit, distributed, in-memory application cache for all kinds of data (CLR objects, rows, XML, Binary data etc.)
Fuse "memory" across machines into a unified cache
Clients can be spread across machines or
processes
Clients Access the Cache as if it
was a large single cache
Cache Layer distributes data
across the various cache
nodes
Where does it fit?
… Application/ Web Tier
Users
Database Cloud Data Tier
Application Application Application
Velocity Access Layer
Velocity Access Layer
Velocity Access Layer
Ve
loci
ty
Serv
ice
Vel
oci
ty
Serv
ice
Vel
oci
ty
Serv
ice
Cache Tier
Why Velocity? 1. Share Data Across Applications
No more sticky routing
Velocity
2. Performance (Quad Core Box)
Operation Throughput Latency
Read 2k 30,000 / sec 3 – 4 ms
Write 2k 18,000 / sec 3 ms
Operation Servers Throughput
Read 2k 1 30,000 / sec
Read 2k 2 58, 600 / sec
Read 2k 3 85, 500 / sec
3. Scale out by adding more boxes
4. High AvailabilityProtect from Web & Cache Server Failure
When Can I Get Velocity?
CTP1
• Teched
CTP2
• PDC 2008
CTP3
• Apr 2009
RTM
• Early Fall 2009
Are customers using it?
Sector Description
Media Cache all Media Files & Metadata; Big Media Provider in US
Content Big Content Provider in Europe
Content MSDN Forums is Live on CTP2
Healthcare One of the large health Care ISV in the world; US
Finance Big Brokerage Firm/Banking; High Performance Computation
Gaming Notifications for Presence;
Enterprise Largest Mainframe Migration to Windows
Manufacturing Manufacturing Process Monitoring
Insurance Session data for insurance purchase
Distributed Caching
Velocity
Overview and Velocity Features
Logical Hierarchy
Deployment Pattern
Access API
Demo – Velocity Installation & Sample Application
Usage Pattern
Demo – Linear Scale
Performance & Availability
Failover
Session cart Integration with ASP.Net
Demo – Availability
Other Features…
Velocity Vision & Futures
Summary
This section..
Velocity Logical Hierarchy
HostPhysical processes hosting Velocity instance.
Cache ItemKey, Payload (Object ), Tags, TTL, Timestamps, Version
Regions Region A
Key Payload Tags
121 xxxx “Toy” “Child”
123 yyyy “Toy” “Chair”..
Velocity Service
Named Cache : Product Catalog
Named Cache : Electronics Inventory
Velocity Service
Velocity Service
Velocity Service
Machine -> Cache Host -> Cache Items -> ObjectsRegions ->Named Cache ->
Named CachesCan span across machines
Defined in the configuration file
RegionsPhysically co-located Container of Cache Items
May be implicit or explicitly created
Deployment
Vel
oci
ty
Serv
ice
Cache Tier
Configuration Store (Can be database, File share, etc.)Stores Global Cache PoliciesStores Current Partitioning Information
Vel
oci
ty
Serv
ice
Vel
oci
ty
Serv
ice
Install Windows Cache Service
Copy Client DLLs and add app/web.config
…
Users
Application/ Web Tier
Application
Velocity Client
Application
Velocity Client
Application
Velocity Client
<configuration><configSections><section name="datacacheClient"
type="Microsoft.Data.Caching.DataCacheClientSection,CacheBaseLibrary" allowLocation="true" allowDefinition="Everywhere"/>
</configSections>
<datacacheClient deployment="routing"><localCache isEnabled="false" sync="TTLBased“
tlValue="300”/><hosts>
<host name=“VELOCITY_SERVER_01" cachePort="22233" cacheHostName="DistributedCacheService"/>
</hosts></datacacheClient>
</configuration>
Velocity API
// Create instance of DataCachefactory (reads appconfig)DataCacheFactory fac = new DataCacheFactory();
// Simple Get/Putcatalog.Put("toy-101", new Toy("Puzzle", .,.));
// From the same or a different clientToy toyObj = (Toy)catalog.Get("toy-101");
// Region based Get/Putcatalog.CreateRegion("toyRegion", true);
// Both toy and toyparts are put in the same region catalog.Put("toy-101", new Toy( .,.), "toyRegion");Catalog.Put("toypart-100", new ToyParts(…), "toyRegion");
Toy toyObj = (Toy)catalog.Get("toy-101", "toyRegion");
// Get a named cache from the factoryDataCache catalog = fac.GetCache("catalogcache");
Usage Pattern – Cache Aside
// Read from CacheToy toyObj = (Toy)catalog.Get("toy-101"); Application
Vel
oci
ty
Serv
ice
// If Not present in the cacheif (toyObj == null){
// Read from backend..toyObj = ReadFromDatabase();
// Populate Cachecatalog.Put("toy-101", toyObj);
}
Demo Output
Single ServerThroughput Increases with Increasing Load
Until Server Saturation
Load
Throughput
Latency
Server 2 AddedThroughput Increases
Latency DecreasesUntil Server Saturation
Server 3 AddedThroughput Increases
Latency Decreases
Demo Layout
Load Test Controller
…
Server 1 Server 2 Server 3
Cache Client 1 Cache Client 2 Cache Client n
Load Controller Spawns Test
Clients
Client Machines run
test
Cache Service Cluster
Partitioned Cache
Application(K2, V2)
Cache2Cache1 Cache3
Primary for (K2,V2)
High Availability
Get(K2)
Primary for (K1,V1) Primary for (K3,V3)
K3, V3
Velocity Client1
Routing Table
K2
, V2
Using the Routing table client routes the PUT to cache2 (primary) node
PUT• Queues the PUT operation • PUTs locally • Propagates operation to
secondary•Returns control
Secondary for (K2,V2), (K3,V3)
K2, V2
K1, V1
K3, V3
Secondary for (K1,V1), (K3,V3)
K3, V3 K1, V1
Secondary for (K1,V1), (K2,V2)
K1, V1
Velocity Client
Routing Table
K2, V2
Replication Agent
Cache4
Primary for (K4,V4)
K4, V4
Secondary for
K1, V1K3, V3
Partition Manager
Global Partition Map
Cache2Cache1 Cache3
Primary for (K2,V2)
Failover
Primary for (K3,V3)
K3, V3
Routing Table
Secondary for
K2, V2
K1, V1
K3, V3
Secondary for
K3, V3 K1, V1
Secondary for
K4, V4K2, V2
K2, V2
Detects Cache 2 failure. Notifies PM (on Cache4)
Replication Agent
ReconfigurationAgent
Local Partition Map
Picks Cache1 as the primary for (K2,V2). Picks Cache 3 as new secondary
PM analyzes the info on secondaries of all primary partitions of Cache2 to elect the primaries.
Cache1 creates new secondary on Cache2 Cache1 initiates reconfiguration.
After reconfig, Cache1 is primary for (K1, V1) and (K2, V2)
Session Cart Integration With ASP.Net
…
Session State stored in Velocity
Vel
oci
ty S
ervi
ce
Vel
oci
ty S
ervi
ce
Load Balance RequestsNo more sticky routing
Application
Velocity SessionStoreProvider
ApplicationVelocity
SessionStoreProvider
Application
Velocity SessionStoreProvider
Vel
oci
ty S
ervi
ce
Scale your Session StoreDynamically
Highly Available
Drop in Velocity SessionStoreProvider
<sessionState mode="Custom" customProvider="SessionStoreProvider"><providers>
<add name="SessionStoreProvider"
type=“Microsoft.Data.Caching.DataCacheSessionStoreProvider, ClientLibrary"
cacheName="<YourNamedCache>"/></providers></sessionState>
Session Store Availability Scenario
Cache ClusterVelocity with
ASP.Net Application
NorthwindApplication
ASP.net Session
Cache
VELOCITY
Velocity SessionStore Provider
NorthwindApplication
ASP.net Session
Cache
VELOCITY
Velocity SessionStore Provider
NorthwindApplication
ASP.net Session
Cache
VELOCITY
Velocity SessionStore Provider
Partitioned Cache
Other Features…Concept of ‘Local cache’
Scalable pull-based Cache Event Notification system
Logging
Provider model; support for file logs & ETW
Administration
Powershell Commandlets
Start and stop cluster or service, configure named caches ..
Monitor Cache statistics
Performance Monitor integration ( by RTM)
…contd.Velocity Service Authentication Security
Custom Security via Firewalls, IPSec, VLANs to secure cache
Domain Based Security Option
Domain Account based Authentication for PS Admin
Authorize at service level
Number of cache servers depend on
Amount of data to cache
Throughput Required
Capacity of the cache server machines
Recommended to have dedicated Velocity Servers
Recap – Before using Velocity
Understand type of data to cacheReference -> Local Caching
Activity -> HA option
Resource -> HA + Locking
Required CharacteristicsPerformance & Availability
Eviction
Security
Number of servers ?Data Size
Throughput Required
Velocity Vision – Cache for all your data
Web Scenarios
• Distributed/Global object cache
• Low latency access
• High scale
• Availability
• Cache for reference and activity data
• Scale IIS/ASP.Netapplications
Enterprise / HPC Scenarios
• LINQ enabled cache
• Co-locate computation and data
• Integrate with HPC server
• Persistence
• Heterogeneous client support.
Software + Services
Scenarios
• Application Cache for cloud Storage
• Integration with SSDS, Windows Azure
• More data services
• BI, Streaming, Reporting
• REST and SOA access
SummaryVelocity Is an In-Memory Distributed Cache for .NET
Roadmap Covers Web, Enterprise, and Cloud Scenarios
Using Velocity Can Help Your App Scale, Perform, and Highly-Available
Performance
Millisecond access; 10s of 1000s of accesses /sec
Scale
10s – 100s of nodes in enterprise; 100s – 1000s in web applications
Almost linear Scale up & Scale Out
Availability
Enterprise and e-commerce data (e.g. shopping cart, inventory)
Resources
Microsoft.com Velocity sitehttp://www.microsoft.com/sql/velocity.aspx
MSDN Data Platform Development Centerhttp://msdn.microsoft.com/data
MSDN Code Galleryhttp://code.msdn.microsoft.com/velocity
MSDN Forums – Access from http://msdn.microsoft.com/data
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS,
IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
