dont call me cache java version
DESCRIPTION
Don't call me cache - Dotan HorovitsTRANSCRIPT
Powering Your Application with XAP
(Using payment processing as an example)
“Don’t call me cache”
Dotan Horovits
Director, Customer Services
Senior solutions architect
About me Dotan Horovits Director, Customer Services Senior system and solutions architect technology evangelist with interest in many things • distributed high-performance systems • big data solutions • cloud technologies • And of course: agile software methodologies • And much more… [email protected] horovits.wordpress.com @horovits
Short Intro to Caching Evolution
Cache In process caching of Key->Value data
structure
Distribute Cache Partitioned cache
nodes
IMDG Partitioned system
of record
In Memory Application Platform
Collocated IMDG and Processing
Cache
Cache is good for repetitive data reads But it is limited in capacity It also doesn’t handle write-heavy scenarios
Short Intro to Caching Evolution
Cache In process caching of Key->Value data
structure
Distribute Cache Partitioned cache
nodes
IMDG Partitioned system
of record
In Memory Application Platform
Collocated IMDG and Processing
Distribute Cache
Allows you to distribute your cache over numerous machines so you get Increased Capacity But it doesn’t support write heavy scenarios It’s also Limited to query by Id What about the rest of your app? - Business logic & messaging??
Short Intro to Caching Evolution
Cache In process caching of Key->Value data
structure
Distribute Cache Partitioned cache
nodes
IMDG Partitioned system
of record
In Memory Application Platform
Collocated IMDG and Processing
IMDG solves these problems! You get increased capacity IMDG is also a System of Record with:
Query APIs
Optimized data access
Data integrity
It solves your write scalability problem
.
Short Intro to Caching Evolution
Cache In process caching of Key->Value data
structure
Distribute Cache Partitioned cache
nodes
IMDG Partitioned system
of record
In Memory Application Platform
Collocated IMDG and Processing
In Memory Application Platform XAP for end to end scaling
Its an IMDG that hosts your Business logic & has messaging services! It Provides Parallel processing of data You get linear scalability You get high availability
How does XAP work?
Here’s What a Payment Authorization Process Looks Like
Payment Authorization
Request Basic Validation User Profile Check
Merchant Profile Check
Payment Authorization
Approved
Write the Payment Object to XAP
Cash Register
Application Payment
User payment Cluster
@SpaceId()
public Long getId() {
return id;
}
@SpaceRouting
public Long getUserId() {
return userId;
}
}
@SpaceIndex public Long getCardId() {
return cardId;
}
}
The primary key of this object in the grid
The grid will use this attribute to route the object to a particular partition
Payment object
A secondary index for query optimization
Write the Payment Object to XAP
Cash Register
Application
User payment Cluster
Payment
@SpaceId
public Long getId() {
return id;
}
@SpaceRouting
public Long getUserId() {
return userId;
}
@SpaceIndex public Long getCardId() {
return cardId;
}
The primary key of this object in the grid
The grid will use this attribute to route the object to a particular partition
Payment object
A Secondary index for query optimization
Write the Payment Object to XAP
Cash Register
Application
User payment Cluster
Payment
Index
@SpaceId
public Long getId() {
return id;
}
@SpaceRouting
public Long getUserId() {
return userId;
}
@SpaceIndex public Long getCardId() {
return cardId;
}
The primary key of this object in the grid
The grid will use this attribute to route the object to a particular partition
Payment object
A Secondary index for query optimization
Let's Talk About Data Model for a Second
User Id (Routing)| User Name|…
Card Id| Card Data | UserID (Routing)…
Transaction ID | CardId | User ID (Routing)…
1 *
1 *
Data Model
Let's Get Back to the Process
Payment Validation
@EventTemplate
public SQLQuery<Payment> getNewPayment()
{
SQLQuery<Payment> query = new SQLQuery<Payment>(Payment.class," paymentStatus = ? ");
query.setParameter(1, Payment.PaymentStatus.New);
return query;
}
Queue
Payment
Validator
Payment Validation
@SpaceDataEvent
public Payment validatePayment(Payment payment)
{
…
}
private boolean basicPaymentValidation(Payment payment, User user, Card card)
{
… }
Queue
Payment
Class PaymentAuthorization{
Status status;
Boolean userCheck;
Boolean vendorCheck;
}
Validator
Payment Authorization
Basic Validation
if (basicPaymentValidation(payment,user,card)) {
gigaSpace.write(userPaymentMsg);
gigaSpace.write(vendorPaymentMsg);
gigaSpace.write(paymentAuthorization);
payment.setPaymentStatus(Payment.PaymentStatus.Processing);
} else {
payment.setPaymentStatus(Payment.PaymentStatus.SuspectedFraud);
}
Class PaymentAuthorization {
Status status;
Boolean userCheck;
Boolean vendorCheck;
}
User Validation
Vendor Validation
Payment Authorization
Queue
Queue
Queue Validator
User Validation
Class PaymentAuthorization{
Enum Status
Boolean userCheck
Boolean vendorCheck
}
Read Message
Process
Queue
Use
r C
red
it C
ard
P
aym
ent
Querying
@SpaceDataEvent
public void validateUser(UserPaymentMsg event) {
… space.readMultiple(new Card(userId))
space.readMultiple(new Payment(userId))
if (valid()) {
IdQuery<PaymentAutjorzation> idQuery = new
IdQuery<PaymentAuthorization>(PaymentAuthoirzation.class, paymentId);
space.change(idQuery, new ChangeSet().set(“userCheck", true));
} else { ... }
}
User Validation
Read Message
Process
Queue
Use
r C
red
it C
ard
P
aym
ent
Class PaymentAuthorization {
Status status;
Boolean userCheck;
Boolean vendorCheck;
}
Querying
@SpaceDataEvent
public void validateUser(UserPaymentMsg event) {
… space.readMultiple(new Card(userId))
space.readMultiple(new Payment(userId))
if (valid()) {
IdQuery<PaymentAutjorzation> idQuery = new
IdQuery<PaymentAuthorization>(PaymentAuthoirzation.class, paymentId);
space.change(idQuery, new ChangeSet().set(“userCheck", true));
} else { ... }
}
User Validation
Read Message
Process
Queue
Use
r C
red
it C
ard
P
aym
ent
@SpaceDataEvent
public void validateUser(UserPaymentMsg event) {
… space.readMultiple(new Card(userId))
space.readMultiple(new Payment(userId))
if (valid()) {
IdQuery<PaymentAutjorzation> idQuery = new
IdQuery<PaymentAuthorization>(PaymentAuthorization.class, paymentId);
space.change(idQuery, new ChangeSet().set(“userCheck", true));
} else { ... }
}
Class PaymentAuthorization {
Status status;
Boolean userCheck;
Boolean vendorCheck;
}
Querying
Vendor Validation
@SpaceDataEvent
public void validateVendor(VendorPaymentMsg vendorPaymentMsg) {
AsyncFuture<Boolean> future = vendorGigaSpace.execute(
new VendorValidationTask(vendorPaymentMsg),
vendorPaymentMsg.getMerchant());
IdQuery<PaymentAuthorization> idQuery = new
IdQuery<PaymentAuthorization>(PaymentAuthorization.class,
vendorPaymentMsg.getPaymentId());
if (future.get()) gigaSpace.change(idQuery, new ChangeSet().
set("vendorCheck", true));
else gigaSpace.change(idQuery, new ChangeSet().
set("vendorCheck”, false));
}
Read Message
Process
Queue
Task
class PaymentAuthoirzation {
Status status;
Boolean userCheck;
Boolean vendorCheck;
}
Vendor Validation
@SpaceDataEvent
public void processVendorPaymentValidation() {
AsyncFuture<Boolean> result= space.execute(
new DistributedTask(new VendorValidationTask(Payment)))
if (result.get()) {
IdQuery<PaymentAutjorzation> idQuery = new
IdQuery<PaymentAuthorization>(PaymentAuthorization.class,
paymentId);
space.change(idQuery, new ChangeSet().set(“vendorCheck”, true));
} else { ... }
}
Payment Cluster
Task
Vendor Cluster
Class PaymentAuthorization {
Status status;
Boolean userCheck;
Boolean vendorCheck;
}
Vendor Validation
public void validateVendor(VendorPaymentMsg vendorPaymentMsg,
GigaSpace gigaSpace) {
AsyncFuture<Boolean> future = vendorGigaSpace.execute(new
VendorValidationTask(vendorPaymentMsg),
vendorPaymentMsg.getMerchant());
IdQuery<PaymentAuthorization> idQuery = new
IdQuery<PaymentAuthorization>(PaymentAuthorization.class,
vendorPaymentMsg.getPaymentId());
if (future.get()) gigaSpace.change(idQuery, new
ChangeSet().set("vendorCheck", true));
else gigaSpace.change(idQuery, new ChangeSet().set("vendorCheck", false));
}
Payment Cluster Vendor Cluster
Task
Class PaymentAuthorization {
Status status;
Boolean userCheck;
Boolean vendorCheck;
}
Vendor Validation
Payment Cluster Vendor Cluster
Task
public void validateVendor(VendorPaymentMsg vendorPaymentMsg,
GigaSpace gigaSpace) {
AsyncFuture<Boolean> future = vendorGigaSpace.execute(new
VendorValidationTask(vendorPaymentMsg),
vendorPaymentMsg.getMerchant());
IdQuery<PaymentAuthorization> idQuery = new
IdQuery<PaymentAuthorization>(PaymentAuthorization.class,
vendorPaymentMsg.getPaymentId());
if (future.get()) gigaSpace.change(idQuery, new
ChangeSet().set("vendorCheck", true));
else gigaSpace.change(idQuery, new ChangeSet().set("vendorCheck", false));
}
Class PaymentAuthorization {
Status status;
Boolean userCheck;
Boolean vendorCheck;
}
Vendor Validation Check
@Override
public Boolean execute() throws Exception {
return validatePayment(vendorPaymentMsg);
}
Task
Dea
ls
Mer
chan
t
Class PaymentAuthorization {
Status status;
Boolean userCheck;
Boolean vendorCheck;
}
Querying
Vendor Validation Check
Task
Dea
ls
Mer
chan
t
@Override
public Boolean execute() throws Exception {
return validatePayment(vendorPaymentMsg);
}
Class PaymentAuthorization {
Status status;
Boolean userCheck;
Boolean vendorCheck;
}
Querying
Vendor Validation
Payment Cluster Vendor Cluster
Callback Task
public void validateVendor(VendorPaymentMsg vendorPaymentMsg,
GigaSpace gigaSpace) {
AsyncFuture<Boolean> future = vendorGigaSpace.execute(new
VendorValidationTask(vendorPaymentMsg),
vendorPaymentMsg.getMerchant());
IdQuery<PaymentAuthorization> idQuery = new
IdQuery<PaymentAuthorization>(PaymentAuthorization.class,
vendorPaymentMsg.getPaymentId());
if (future.get()) gigaSpace.change(idQuery, new
ChangeSet().set("vendorCheck", true));
else gigaSpace.change(idQuery, new ChangeSet().set("vendorCheck", false));
}
Class PaymentAuthorization {
Status status;
Boolean userCheck;
Boolean vendorCheck;
}
Yey! Payment Has Been Authorized
@EventTemplate
public PaymentAuthorization getNewPayment() {
return new PaymentAuthorization(null, true, true, PaymentAuthorizationStatus.New);
}
Queue
Payment Authorization
Yey! Payment Has Been Authorized
@SpaceDataEvent
public void completePaymentValidation(PaymentAuthorization paymentAuthorization) {
Payment payment = gigaSpace.readById(Payment.class,paymentAuthorization.getPaymentId());
payment.setPaymentStatus(Payment.PaymentStatus.Closed);
gigaSpace.write(payment);
paymentAuthorization.setPaymentAuthorizationStatus(
PaymentAuthorization.PaymentAuthorizationStatus.Done);
}
Queue
Payment Authorization
What’s next?
• Multi-site deployment?
• Persistence to DB for auditing?
• Using Scala? .NET? scripts?
• Schema evolution?
• Running our clusters on the cloud?
• Etc.
Q&A
Source code (Java) publically available on GitHub: https://github.com/Gigaspaces/fraud-detection-example