developing polyglot persistence applications (springone china 2012)

90
Developing polyglot persistence applications Chris Richardson, Author of POJOs in Action, Founder of the original CloudFoundry.com @crichardson [email protected] http://plainoldobjects.com

Upload: chris-richardson

Post on 10-May-2015

3.235 views

Category:

Technology


5 download

TRANSCRIPT

Page 1: Developing polyglot persistence applications (SpringOne China 2012)

Developing polyglot persistence applications

Chris Richardson, Author of POJOs in Action, Founder of the original CloudFoundry.com @crichardson [email protected] http://plainoldobjects.com

Page 2: Developing polyglot persistence applications (SpringOne China 2012)

Presentation goal

The benefits and drawbacks of polyglot persistence

and

How to design applications that use this approach

Page 3: Developing polyglot persistence applications (SpringOne China 2012)

About Chris

Page 4: Developing polyglot persistence applications (SpringOne China 2012)

(About Chris)

Page 5: Developing polyglot persistence applications (SpringOne China 2012)

About Chris()

Page 6: Developing polyglot persistence applications (SpringOne China 2012)

About Chris

Page 7: Developing polyglot persistence applications (SpringOne China 2012)

About Chris

http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/

Page 8: Developing polyglot persistence applications (SpringOne China 2012)

vmc push About-Chris

Developer Advocate

Signup at http://cloudfoundry.com

Page 9: Developing polyglot persistence applications (SpringOne China 2012)

Agenda

• Why polyglot persistence?

• Using Redis as a cache

• Optimizing queries using Redis materialized views

• Synchronizing MySQL and Redis

• Tracking changes to entities

• Using a modular asynchronous architecture

Page 10: Developing polyglot persistence applications (SpringOne China 2012)

Food to Go

• Take-out food delivery service

• “Launched” in 2006

Page 11: Developing polyglot persistence applications (SpringOne China 2012)

Food To Go Architecture

Order taking Restaurant Management

MySQL Database

CONSUMER RESTAURANT OWNER

Page 12: Developing polyglot persistence applications (SpringOne China 2012)

Success ⇒ Growth challenges

• Increasing traffic

• Increasing data volume

• Distribute across a few data centers

• Increasing domain model complexity

Page 13: Developing polyglot persistence applications (SpringOne China 2012)

Limitations of relational databases

• Scalability

• Distribution

• Schema updates

• O/R impedance mismatch

• Handling semi-structured data

Page 14: Developing polyglot persistence applications (SpringOne China 2012)

Solution: Spend Money

http://upload.wikimedia.org/wikipedia/commons/e/e5/Rising_Sun_Yacht.JPG

OR

http://www.trekbikes.com/us/en/bikes/road/race_performance/madone_5_series/madone_5_2/#

Page 15: Developing polyglot persistence applications (SpringOne China 2012)

Solution: Use NoSQL

Benefits

• Higher performance

• Higher scalability

• Richer data-model

• Schema-less

Drawbacks

• Limited transactions

• Limited querying

• Relaxed consistency

• Unconstrained data

Page 16: Developing polyglot persistence applications (SpringOne China 2012)

Example NoSQL Databases

Database Key features

Cassandra Extensible column store, very scalable, distributed

Neo4j Graph database

MongoDB Document-oriented, fast, scalable

Redis Key-value store, very fast

http://nosql-database.org/ lists 122+ NoSQL databases

Page 17: Developing polyglot persistence applications (SpringOne China 2012)

• Advanced key-value store

• Very fast, e.g. 100K reqs/sec

• Optional persistence

• Transactions with optimistic locking

• Master-slave replication

• Sharding using client-side consistent hashing

Redis

K1 V1

K2 V2

... ...

Page 18: Developing polyglot persistence applications (SpringOne China 2012)

Sorted sets

myset a

5.0

b

10.0

Key Value

ScoreMembers are sorted by score

Page 19: Developing polyglot persistence applications (SpringOne China 2012)

Adding members to a sorted set

Redis Server

zadd myset 5.0 a myset a

5.0

Key Score Value

Page 20: Developing polyglot persistence applications (SpringOne China 2012)

Adding members to a sorted set

Redis Server

zadd myset 10.0 b myset a

5.0

b

10.0

Page 21: Developing polyglot persistence applications (SpringOne China 2012)

Adding members to a sorted set

Redis Server

zadd myset 1.0 c myset a

5.0

b

10.0

c

1.0

Page 22: Developing polyglot persistence applications (SpringOne China 2012)

Retrieving members by index range

Redis Server

zrange myset 0 1

myset a

5.0

b

10.0

c

1.0ac

Key StartIndex

EndIndex

Page 23: Developing polyglot persistence applications (SpringOne China 2012)

Retrieving members by score

Redis Server

zrangebyscore myset 1 6

myset a

5.0

b

10.0

c

1.0ac

Key Min value

Max value

Page 24: Developing polyglot persistence applications (SpringOne China 2012)

Redis use cases• Replacement for Memcached

• Session state

• Cache of data retrieved from system of record (SOR)

• Replica of SOR for queries needing high-performance

• Handling tasks that overload an RDBMS

• Hit counts - INCR

• Most recent N items - LPUSH and LTRIM

• Randomly selecting an item – SRANDMEMBER

• Queuing – Lists with LPOP, RPUSH, ….

• High score tables – Sorted sets and ZINCRBY

• …

Page 25: Developing polyglot persistence applications (SpringOne China 2012)

Redis is great but there are tradeoffs

• Low-level query language: PK-based access only

• Limited transaction model:

• Read first and then execute updates as batch

• Difficult to compose code

• Data must fit in memory

• Single-threaded server : run multiple with client-side sharding

• Missing features such as access control, ...

Page 26: Developing polyglot persistence applications (SpringOne China 2012)

And don’t forget:

An RDBMS is fine for many applications

Page 27: Developing polyglot persistence applications (SpringOne China 2012)

The future is polyglot

IEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg

e.g. Netflix• RDBMS• SimpleDB• Cassandra• Hadoop/Hbase

Page 28: Developing polyglot persistence applications (SpringOne China 2012)

Agenda

• Why polyglot persistence?

• Using Redis as a cache

• Optimizing queries using Redis materialized views

• Synchronizing MySQL and Redis

• Tracking changes to entities

• Using a modular asynchronous architecture

Page 29: Developing polyglot persistence applications (SpringOne China 2012)

Increase scalability by caching

Order taking Restaurant Management

MySQL Database

CONSUMER RESTAURANT OWNER

Cache

Page 30: Developing polyglot persistence applications (SpringOne China 2012)

Caching Options

• Where:

• Hibernate 2nd level cache

• Explicit calls from application code

• Caching aspect

• Cache technologies: Ehcache, Memcached, Infinispan, ...

Redis is also an option

Page 31: Developing polyglot persistence applications (SpringOne China 2012)

Using Redis as a cache

• Spring 3.1 cache abstraction

• Annotations specify which methods to cache

• CacheManager - pluggable back-end cache

• Spring Data for Redis

• Simplifies the development of Redis applications

• Provides RedisTemplate (analogous to JdbcTemplate)

• Provides RedisCacheManager

Page 32: Developing polyglot persistence applications (SpringOne China 2012)

Using Spring 3.1 Caching@Servicepublic class RestaurantManagementServiceImpl implements RestaurantManagementService {

private final RestaurantRepository restaurantRepository;

@Autowired public RestaurantManagementServiceImpl(RestaurantRepository restaurantRepository) { this.restaurantRepository = restaurantRepository; } @Override public void add(Restaurant restaurant) { restaurantRepository.add(restaurant); }

@Override @Cacheable(value = "Restaurant") public Restaurant findById(int id) { return restaurantRepository.findRestaurant(id); }

@Override @CacheEvict(value = "Restaurant", key="#restaurant.id") public void update(Restaurant restaurant) { restaurantRepository.update(restaurant); }

Cache result

Evict from cache

Page 33: Developing polyglot persistence applications (SpringOne China 2012)

Configuring the Redis Cache Manager

<cache:annotation-driven />

<bean id="cacheManager" class="org.springframework.data.redis.cache.RedisCacheManager" > <constructor-arg ref="restaurantTemplate"/> </bean>

Enables caching

Specifies CacheManager implementation

The RedisTemplate used to access Redis

Page 34: Developing polyglot persistence applications (SpringOne China 2012)

TimeRange MenuItem

Domain object to key-value mapping?

Restaurant

TimeRange MenuItem

ServiceArea

K1 V1

K2 V2

... ...

Page 35: Developing polyglot persistence applications (SpringOne China 2012)

RedisTemplate

• Analogous to JdbcTemplate

• Encapsulates boilerplate code, e.g. connection management

• Maps Java objects ⇔ Redis byte[]’s

Page 36: Developing polyglot persistence applications (SpringOne China 2012)

Serializers: object ⇔ byte[]

• RedisTemplate has multiple serializers

• DefaultSerializer - defaults to JdkSerializationRedisSerializer

• KeySerializer

• ValueSerializer

• HashKeySerializer

• HashValueSerializer

Page 37: Developing polyglot persistence applications (SpringOne China 2012)

Serializing a Restaurant as JSON@Configurationpublic class RestaurantManagementRedisConfiguration {

@Autowired private RestaurantObjectMapperFactory restaurantObjectMapperFactory; private JacksonJsonRedisSerializer<Restaurant> makeRestaurantJsonSerializer() { JacksonJsonRedisSerializer<Restaurant> serializer =

new JacksonJsonRedisSerializer<Restaurant>(Restaurant.class); ... return serializer; }

@Bean @Qualifier("Restaurant") public RedisTemplate<String, Restaurant> restaurantTemplate(RedisConnectionFactory factory) { RedisTemplate<String, Restaurant> template = new RedisTemplate<String, Restaurant>(); template.setConnectionFactory(factory); JacksonJsonRedisSerializer<Restaurant> jsonSerializer = makeRestaurantJsonSerializer(); template.setValueSerializer(jsonSerializer); return template; }

} Serialize restaurants using Jackson JSON

Page 38: Developing polyglot persistence applications (SpringOne China 2012)

Caching with Redis

Order taking Restaurant Management

MySQL Database

CONSUMER RESTAURANT OWNER

RedisCacheFirst Second

Page 39: Developing polyglot persistence applications (SpringOne China 2012)

Agenda

• Why polyglot persistence?

• Using Redis as a cache

• Optimizing queries using Redis materialized views

• Synchronizing MySQL and Redis

• Tracking changes to entities

• Using a modular asynchronous architecture

Page 40: Developing polyglot persistence applications (SpringOne China 2012)

Finding available restaurants

Available restaurants =Serve the zip code of the delivery address

AND Are open at the delivery time

public interface AvailableRestaurantRepository {

List<AvailableRestaurant> findAvailableRestaurants(Address deliveryAddress, Date deliveryTime); ...}

Page 41: Developing polyglot persistence applications (SpringOne China 2012)

Food to Go – Domain model (partial)class Restaurant { long id; String name; Set<String> serviceArea; Set<TimeRange> openingHours; List<MenuItem> menuItems;}

class MenuItem { String name; double price;}

class TimeRange { long id; int dayOfWeek; int openTime; int closeTime;}

Page 42: Developing polyglot persistence applications (SpringOne China 2012)

Database schemaID Name …

1 Ajanta

2 Montclair Eggshop

Restaurant_id zipcode

1 94707

1 94619

2 94611

2 94619

Restaurant_id dayOfWeek openTime closeTime

1 Monday 1130 1430

1 Monday 1730 2130

2 Tuesday 1130 …

RESTAURANT table

RESTAURANT_ZIPCODE table

RESTAURANT_TIME_RANGE table

Page 43: Developing polyglot persistence applications (SpringOne China 2012)

Finding available restaurants on Monday, 6.15pm for 94619 zipcode

Straightforward three-way join

select r.*from restaurant r inner join restaurant_time_range tr on r.id =tr.restaurant_id inner join restaurant_zipcode sa on r.id = sa.restaurant_id where ’94619’ = sa.zip_code and tr.day_of_week=’monday’ and tr.openingtime <= 1815 and 1815 <= tr.closingtime

Page 44: Developing polyglot persistence applications (SpringOne China 2012)

How to scale queries?

Page 45: Developing polyglot persistence applications (SpringOne China 2012)

Option #1: Query caching

• [ZipCode, DeliveryTime] ⇨ list of available restaurants

BUT

• Long tail queries

• Update restaurant ⇨ Flush entire cache

Ineffective

Page 46: Developing polyglot persistence applications (SpringOne China 2012)

Option #2: Master/Slave replication

MySQL Master

MySQL Slave 1

MySQL Slave 2

MySQL Slave N

Writes Consistent reads

Queries(Inconsistent reads)

Page 47: Developing polyglot persistence applications (SpringOne China 2012)

Master/Slave replication

• Mostly straightforward

BUT

• Assumes that SQL query is efficient

• Complexity of administration of slaves

• Doesn’t scale writes

Page 48: Developing polyglot persistence applications (SpringOne China 2012)

Option #3: Redis materialized views

Order taking Restaurant Management

MySQL Database

CONSUMER RESTAURANT OWNER

CacheRedis

Systemof RecordCopy

findAvailable()update()

Page 49: Developing polyglot persistence applications (SpringOne China 2012)

BUT how to implement findAvailableRestaurants() with Redis?!

?select r.*from restaurant r inner join restaurant_time_range tr on r.id =tr.restaurant_id inner join restaurant_zipcode sa on r.id = sa.restaurant_id where ’94619’ = sa.zip_code

and tr.day_of_week=’monday’

and tr.openingtime <= 1815

and 1815 <= tr.closingtime

K1 V1

K2 V2

... ...

Page 50: Developing polyglot persistence applications (SpringOne China 2012)

Where we need to be

ZRANGEBYSCORE myset 1 6

select value,scorefrom sorted_setwhere key = ‘myset’ and score >= 1 and score <= 6

=

key value score

sorted_set

Page 51: Developing polyglot persistence applications (SpringOne China 2012)

We need to denormalize

Think materialized view

Page 52: Developing polyglot persistence applications (SpringOne China 2012)

Simplification #1: Denormalization

Restaurant_id Day_of_week Open_time Close_time Zip_code

1 Monday 1130 1430 94707

1 Monday 1130 1430 94619

1 Monday 1730 2130 94707

1 Monday 1730 2130 94619

2 Monday 0700 1430 94619

SELECT restaurant_id FROM time_range_zip_code WHERE day_of_week = ‘Monday’ AND zip_code = 94619 AND 1815 < close_time AND open_time < 1815

Simpler query:§ No joins§ Two = and two <

Page 53: Developing polyglot persistence applications (SpringOne China 2012)

Simplification #2: Application filtering

SELECT restaurant_id, open_time FROM time_range_zip_code WHERE day_of_week = ‘Monday’ AND zip_code = 94619 AND 1815 < close_time AND open_time < 1815

Even simpler query• No joins• Two = and one <

Page 54: Developing polyglot persistence applications (SpringOne China 2012)

Simplification #3: Eliminate multiple =’s with concatenation

SELECT restaurant_id, open_time FROM time_range_zip_code WHERE zip_code_day_of_week = ‘94619:Monday’ AND 1815 < close_time

Restaurant_id Zip_dow Open_time Close_time

1 94707:Monday 1130 1430

1 94619:Monday 1130 1430

1 94707:Monday 1730 2130

1 94619:Monday 1730 2130

2 94619:Monday 0700 1430

key

range

Page 55: Developing polyglot persistence applications (SpringOne China 2012)

Simplification #4: Eliminate multiple RETURN VALUES with concatenation

SELECT open_time_restaurant_id, FROM time_range_zip_code WHERE zip_code_day_of_week = ‘94619:Monday’ AND 1815 < close_time

zip_dow open_time_restaurant_id close_time

94707:Monday 1130_1 1430

94619:Monday 1130_1 1430

94707:Monday 1730_1 2130

94619:Monday 1730_1 2130

94619:Monday 0700_2 1430

...

Page 56: Developing polyglot persistence applications (SpringOne China 2012)

zip_dow open_time_restaurant_id close_time

94707:Monday 1130_1 1430

94619:Monday 1130_1 1430

94707:Monday 1730_1 2130

94619:Monday 1730_1 2130

94619:Monday 0700_2 1430

...

Sorted Set [ Entry:Score, …]

[1130_1:1430, 1730_1:2130]

[0700_2:1430, 1130_1:1430, 1730_1:2130]

Using a Redis sorted set as an index

Key

94619:Monday

94707:Monday

94619:Monday 0700_2 1430

Page 57: Developing polyglot persistence applications (SpringOne China 2012)

Querying with ZRANGEBYSCORE

ZRANGEBYSCORE 94619:Monday 1815 2359è

{1730_1}

1730 is before 1815 è Ajanta is open

Delivery timeDelivery zip and day

Sorted Set [ Entry:Score, …]

[1130_1:1430, 1730_1:2130]

[0700_2:1430, 1130_1:1430, 1730_1:2130]

Key

94619:Monday

94707:Monday

Page 58: Developing polyglot persistence applications (SpringOne China 2012)

Adding a Restaurant

Text

@Componentpublic class AvailableRestaurantRepositoryImpl implements AvailableRestaurantRepository {

@Override public void add(Restaurant restaurant) { addRestaurantDetails(restaurant); addAvailabilityIndexEntries(restaurant); }

private void addRestaurantDetails(Restaurant restaurant) { restaurantTemplate.opsForValue().set(keyFormatter.key(restaurant.getId()), restaurant); } private void addAvailabilityIndexEntries(Restaurant restaurant) { for (TimeRange tr : restaurant.getOpeningHours()) { String indexValue = formatTrId(restaurant, tr); int dayOfWeek = tr.getDayOfWeek(); int closingTime = tr.getClosingTime(); for (String zipCode : restaurant.getServiceArea()) { redisTemplate.opsForZSet().add(closingTimesKey(zipCode, dayOfWeek), indexValue,

closingTime); } } }

key member

score

Store as JSON

Page 59: Developing polyglot persistence applications (SpringOne China 2012)

Finding available Restaurants

@Componentpublic class AvailableRestaurantRepositoryImpl implements AvailableRestaurantRepository { @Override public List<AvailableRestaurant>

findAvailableRestaurants(Address deliveryAddress, Date deliveryTime) { String zipCode = deliveryAddress.getZip(); int dayOfWeek = DateTimeUtil.dayOfWeek(deliveryTime); int timeOfDay = DateTimeUtil.timeOfDay(deliveryTime); String closingTimesKey = closingTimesKey(zipCode, dayOfWeek);

Set<String> trsClosingAfter = redisTemplate.opsForZSet().rangeByScore(closingTimesKey, timeOfDay, 2359);

Set<String> restaurantIds = new HashSet<String>(); for (String tr : trsClosingAfter) { String[] values = tr.split("_"); if (Integer.parseInt(values[0]) <= timeOfDay) restaurantIds.add(values[1]); } Collection<String> keys = keyFormatter.keys(restaurantIds); return availableRestaurantTemplate.opsForValue().multiGet(keys); }

Find those that close after

Filter out those that open after

Retrieve open restaurants

Page 60: Developing polyglot persistence applications (SpringOne China 2012)

Sorry Ted!

http://en.wikipedia.org/wiki/Edgar_F._Codd

Page 61: Developing polyglot persistence applications (SpringOne China 2012)

Agenda

• Why polyglot persistence?

• Using Redis as a cache

• Optimizing queries using Redis materialized views

• Synchronizing MySQL and Redis

• Tracking changes to entities

• Using a modular asynchronous architecture

Page 62: Developing polyglot persistence applications (SpringOne China 2012)

MySQL & Redis need to be consistent

Page 63: Developing polyglot persistence applications (SpringOne China 2012)

Two-Phase commit is not an option

• Redis does not support it

• Even if it did, 2PC is best avoided

http://www.infoq.com/articles/ebay-scalability-best-practices

Page 64: Developing polyglot persistence applications (SpringOne China 2012)

AtomicConsistentIsolatedDurable

Basically AvailableSoft stateEventually consistent

BASE: An Acid Alternative http://queue.acm.org/detail.cfm?id=1394128

Page 65: Developing polyglot persistence applications (SpringOne China 2012)

Updating Redis #FAILbegin MySQL transaction update MySQL update Redisrollback MySQL transaction

Redis has updateMySQL does not

begin MySQL transaction update MySQLcommit MySQL transaction<<system crashes>> update Redis

MySQL has updateRedis does not

Page 66: Developing polyglot persistence applications (SpringOne China 2012)

Updating Redis reliablyStep 1 of 2

begin MySQL transactionupdate MySQLqueue CRUD event in MySQL

commit transaction

ACID

Event IdOperation: Create, Update, DeleteNew entity state, e.g. JSON

Page 67: Developing polyglot persistence applications (SpringOne China 2012)

Updating Redis reliably Step 2 of 2

for each CRUD event in MySQL queue get next CRUD event from MySQL queue

If CRUD event is not duplicate then Update Redis (incl. eventId)end ifbegin MySQL transaction

mark CRUD event as processedcommit transaction

Page 68: Developing polyglot persistence applications (SpringOne China 2012)

ID JSON processed?

ENTITY_CRUD_EVENT

EntityCrudEventProcessor

RedisUpdater

apply(event)

Step 1 Step 2

EntityCrudEventRepository

INSERT INTO ...

Timer

SELECT ... FROM ...

Redis

Page 69: Developing polyglot persistence applications (SpringOne China 2012)

Updating Redis

WATCH restaurant:lastSeenEventId:≪restaurantId≫

Optimistic locking

Duplicate detection

lastSeenEventId = GET restaurant:lastSeenEventId:≪restaurantId≫

if (lastSeenEventId >= eventId) return;

Transaction

MULTI SET restaurant:lastSeenEventId:≪restaurantId≫ eventId

... update the restaurant data...

EXEC

Page 70: Developing polyglot persistence applications (SpringOne China 2012)

Agenda

• Why polyglot persistence?

• Using Redis as a cache

• Optimizing queries using Redis materialized views

• Synchronizing MySQL and Redis

• Tracking changes to entities

• Using a modular asynchronous architecture

Page 71: Developing polyglot persistence applications (SpringOne China 2012)

How do we generate CRUD events?

Page 72: Developing polyglot persistence applications (SpringOne China 2012)

Change tracking options

• Explicit code

• Hibernate event listener

• Service-layer aspect

• CQRS/Event-sourcing

Page 73: Developing polyglot persistence applications (SpringOne China 2012)

ID JSON processed?

ENTITY_CRUD_EVENT

EntityCrudEventRepository

HibernateEventListener

Page 74: Developing polyglot persistence applications (SpringOne China 2012)

Hibernate event listenerpublic class ChangeTrackingListener implements PostInsertEventListener, PostDeleteEventListener, PostUpdateEventListener { @Autowired private EntityCrudEventRepository entityCrudEventRepository; private void maybeTrackChange(Object entity, EntityCrudEventType eventType) { if (isTrackedEntity(entity)) { entityCrudEventRepository.add(new EntityCrudEvent(eventType, entity)); } }

@Override public void onPostInsert(PostInsertEvent event) { Object entity = event.getEntity(); maybeTrackChange(entity, EntityCrudEventType.CREATE); }

@Override public void onPostUpdate(PostUpdateEvent event) { Object entity = event.getEntity(); maybeTrackChange(entity, EntityCrudEventType.UPDATE); }

@Override public void onPostDelete(PostDeleteEvent event) { Object entity = event.getEntity(); maybeTrackChange(entity, EntityCrudEventType.DELETE); }

Page 75: Developing polyglot persistence applications (SpringOne China 2012)

Agenda

• Why polyglot persistence?

• Using Redis as a cache

• Optimizing queries using Redis materialized views

• Synchronizing MySQL and Redis

• Tracking changes to entities

• Using a modular asynchronous architecture

Page 76: Developing polyglot persistence applications (SpringOne China 2012)

Original architecture

WAR

RestaurantManagement

...

Page 77: Developing polyglot persistence applications (SpringOne China 2012)

Drawbacks of this monolithic architecture

• Obstacle to frequent deployments

• Overloads IDE and web container

• Obstacle to scaling development

• Technology lock-in

WAR

RestaurantManagement

...

Page 78: Developing polyglot persistence applications (SpringOne China 2012)

Need a more modular architecture

Page 79: Developing polyglot persistence applications (SpringOne China 2012)

Using a message broker

Asynchronous is preferred

JSON is fashionable but binary format is more efficient

Page 80: Developing polyglot persistence applications (SpringOne China 2012)

Modular architecture

Order taking Restaurant Management

MySQL Database

RedisCache

Redis RabbitMQ

CONSUMER RESTAURANT OWNER

Event Publisher

Timer

Page 81: Developing polyglot persistence applications (SpringOne China 2012)

Benefits of a modular asynchronous architecture

• Scales development: develop, deploy and scale each service independently

• Redeploy UI frequently/independently

• Improves fault isolation

• Eliminates long-term commitment to a single technology stack

• Message broker decouples producers and consumers

Page 82: Developing polyglot persistence applications (SpringOne China 2012)

Step 2 of 2

for each CRUD event in MySQL queue get next CRUD event from MySQL queue

Publish persistent message to RabbitMQbegin MySQL transaction

mark CRUD event as processedcommit transaction

Page 83: Developing polyglot persistence applications (SpringOne China 2012)

Message flowEntityCrudEvent

Processor

RedisUpdater

RABBITMQ

AvailableRestaurantManagementService

REDIS

Spring Integration glue code

Page 84: Developing polyglot persistence applications (SpringOne China 2012)

RedisUpdater ⇒ AMQP<beans>

<int:gateway id="redisUpdaterGateway" service-interface="net...RedisUpdater" default-request-channel="eventChannel" />

<int:channel id="eventChannel"/>

<int:object-to-json-transformer input-channel="eventChannel" output-channel="amqpOut"/>

<int:channel id="amqpOut"/>

<amqp:outbound-channel-adapter channel="amqpOut" amqp-template="rabbitTemplate" routing-key="crudEvents" exchange-name="crudEvents" />

</beans>

Creates proxy

Page 85: Developing polyglot persistence applications (SpringOne China 2012)

AMQP ⇒ Available...Service<beans> <amqp:inbound-channel-adapter channel="inboundJsonEventsChannel" connection-factory="rabbitConnectionFactory" queue-names="crudEvents"/>

<int:channel id="inboundJsonEventsChannel"/> <int:json-to-object-transformer input-channel="inboundJsonEventsChannel" type="net.chrisrichardson.foodToGo.common.JsonEntityCrudEvent" output-channel="inboundEventsChannel"/> <int:channel id="inboundEventsChannel"/> <int:service-activator input-channel="inboundEventsChannel" ref="availableRestaurantManagementServiceImpl" method="processEvent"/></beans>

Invokes service

Page 86: Developing polyglot persistence applications (SpringOne China 2012)

Summary

• Each SQL/NoSQL database = set of tradeoffs

• Polyglot persistence: leverage the strengths of SQL and NoSQL databases

• Use Redis as a distributed cache

• Store denormalized data in Redis for fast querying

• Reliable database synchronization required

Page 87: Developing polyglot persistence applications (SpringOne China 2012)

Questions?

@crichardson [email protected] http://plainoldobjects.com

Sign up for CloudFoundry.com

Page 88: Developing polyglot persistence applications (SpringOne China 2012)

88

Cloud Foundry 启动营在www.cloudfoundry.com注册账号并成功上传应用程序,即可于12月8日中午后凭账号ID和应用URL到签到处换取Cloud Foundry主题卫衣一件。

Page 89: Developing polyglot persistence applications (SpringOne China 2012)

89

iPhone5 等你拿第二天大会结束前,请不要提前离开,将填写完整的意见反馈表投到签到处的抽奖箱内,即可参与“iPhone5”抽奖活动。

Page 90: Developing polyglot persistence applications (SpringOne China 2012)

90

Birds of a Feather 专家面对面所有讲师都会在课程结束后,到紫兰厅与来宾讨论课程上的问题