imdgs an essential part of your architecture. about me [email protected]
TRANSCRIPT
IMDGsAn essential part of your architecture
What is IMDG?
In short: it’s performance and scalability enabling solution
Cyber Monday
Solution
Client App
write
No SQL
node #1
node #2
node #3
node #4
node #5
read throughCache
• Low latency
NoSQL
• Scalability
• Fault tolerance
• Data management
NOT CONSISTENT
Grid SolutionGrid
• Low latency
• Fault tolerance
• Data management
• Elasticity
• ACID transactions
Client App
write
node #1
node #5
node #2
node #4
node #3
node #1
node #2
node #3
node #4
node #5
read
writebehindpre-load
Cache
• Key-value storage which works with domain objects, may support a document API• Eviction and invalidation policies• Read and write strategies: read through, write through, read-ahead,
write behind, Change Data Capture• Local cache and Local view• Date pre-load, recovery from disk during cold start and after
simultaneous node failure
Data
• Query language, indexes• parallel query and "map/reduce"
Clients & API
• API and client library for different programming languages• Transparent proxy, grid• Pluggable connectors for databases• JDBC, JMS, Memcached connectivity
Infrastructure
• Easy clustering, zero time deployment• Monitoring, management and Self healing• Configurable topology with Sync/Async Backups• SLA based elastic deployments• Security for connecting a node to space and client to space
Tool for In-Memory
SOA is not scalable
Processing
• Task Execution, Processing Grid and (Massive) Parallel Executions• Remote Services, Collocated Execution• Complex Event Processing• Pooling Container• Notify Container
Case Study
16
Authorization ServerThree steps:
• Validate digital signature using stored Access Token
• Validate message nonce uniqueness for the last 15 minutes
• Validate URL access by checking of all ACLs configured for the client
Client Web Application Authorization Service
GET (resource, auth_header)
AuthorizeAcess (resource, auth_header)
Validate ACL
Is Nonce Unique for the Client
auth_header:
· token_id· signature· nonce
Authentication Status: 200/401/403
Space
getToken(token_id)
getAccssControlList(client_id)
Loop ~ 25 times
Validate Signature
getNonce(nonce)
writeNonce(nonce)
Authentication Status: 200/401/403
Challanges
• 50 ms SLA• Strong Consistency• Protect any requests to any service• Durable
18
Architecture with CacheMemcachedApplication Farm
1..n 1..n
MongoDB Shard
1..n
19
Architecture with GridAll elements of the architecture are fully provided by GigaSpaces XAP platform
My SQL
Backup SpacePrimary Space
Service
Apache LB Backup SpacePrimary Space
Service
Backup SpacePrimary Space
Service
Elastic Web Elastic Space with Back-up Async Persistance
Web PU
Local Cache
Web PU
Local Cache
Web PU
Local Cache
Mirror
20
Performance
Architecture App Servers, nodes
Spaces / Memcached
, nodesUsers Throughput,
rq/sAverage,
msMedian
, ms90% Line,
ms
1 Node (Users ratio 3.75, Throughput ratio 3.57)
Grid 1 1 45 579.93 41 41 47
Cache 1 0 12 162.70 34 30 45
6 Nodes (Users ratio 1.81, Throughput ratio 2.32)
Grid 6 6 136 2,406.97 32 24 53Cache 4 2 75 1,037.15 36 31 54
21
Scalability
2 3 4 5 60
500
1000
1500
2000
2500
987
1720
2047
163
545665 695
1037
Grid Cache
Areas• Low latency applications• Big Data 3 V (Verity, Volume, Velocity)• Real time big data analysis (Fast Data)• Stream processing• Complex Event Processing