redis adaptor for apache geode
TRANSCRIPT
![Page 1: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/1.jpg)
Redis Adaptor
Swapnil Bawaskar@sbawaskar
(incubating)
![Page 2: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/2.jpg)
• Redis Primer • Need • Implementation • Advantages • Roadmap
Agenda
2
![Page 3: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/3.jpg)
Redis Primer
3
![Page 4: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/4.jpg)
Data Structures Supported • Strings • List • Hashes • Sets • SortedSets • HyperLogLog
Redis Primer
4
![Page 5: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/5.jpg)
Redis Commands
Redis Primer
5
redis>SETkey1"value"OKredis>LPUSHkey2"value"(integer)1redis>SADDkey3"value"(integer)1redis>TYPEkey1stringredis>TYPEkey2listredis>TYPEkey3setredis>
In Redis commands • key refers to a data structure name • Command determines the data structure type
![Page 6: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/6.jpg)
Redis Cluster
6
![Page 7: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/7.jpg)
Back in 2014
7
![Page 8: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/8.jpg)
Y U No
8
![Page 9: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/9.jpg)
Geode Implementation
9
![Page 10: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/10.jpg)
Redis Adaptor
10
RedisClient
RedisClient
RedisClient
RedisClient
Redis Server
![Page 11: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/11.jpg)
Redis Adaptor
11
RedisClient
RedisClient
RedisClient
RedisClient
Geode Server
Drop-in Replacement
![Page 12: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/12.jpg)
Start Using GFSH
Redis Adaptor - Usage
12
gfsh>startserver--name=server1--redis-port=11211--redis-bind-address=localhost
Creates a region type PARTITION by default • Change Using system property gemfireredis.regiontype
gfsh>startserver--name=server1--redis-port=11211--J=-Dgemfireredis.regiontype=PARTITION_PERSISTENT
![Page 13: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/13.jpg)
• Supports all Redis data structures • Each data structure (except Strings) is
backed by a Geode Partitioned Region • which is a horizontally scalable distributed
ConcurrentHashMap • We create just one Partitioned Region for
storing all Strings • Uses Geode’s OQL and indexes
Redis Adaptor - Implementation
13
![Page 14: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/14.jpg)
• One Partitioned Region for Each List
• Track “head” and “tail” position in the Partitioned Region itself as special entries
Future Exploration • Embed the position in “Value” • Place an index on the position,
should help with List Scan
Redis Adaptor - List
14
user dataposition
Key Value
![Page 15: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/15.jpg)
• One Partitioned Region for Each Sorted Set
• key is user data, value is user provided score
• Index on score
Redis Adaptor - Sorted Set
15
scoreuser data
Key Value
![Page 16: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/16.jpg)
Advantages
16
![Page 17: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/17.jpg)
• Scalability • Server side stored procedures • Network Partitioning • WAN replication
Advantages
17
![Page 18: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/18.jpg)
• Scale Vertically • Redis server is single threaded • Geode server supports high concurrency • Utilize CPUs on big boxes without having to
manage more instances • Scale data structures - Horizontally
Scalability
18
![Page 19: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/19.jpg)
• Redis lost 56% writes during Network Partition*
• Geode servers fence themselves
Network Partition
19
* Aphyr - https://aphyr.com/posts/283-jepsen-redis
![Page 20: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/20.jpg)
• Data can be replicated to other clusters over WAN
WAN Replication
20
WAN
![Page 21: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/21.jpg)
Roadmap
21
![Page 22: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/22.jpg)
• Implement “All” Commands • String: SETRANGE (overwrites part of string) • List: BLPOP, BRPOPLPUSH (Block/insert into another) • Sorted Set: ZINTERSTORE (intersect multiple Sorted
Sets and store result in another) • etc.
• Performance benchmark • Optimize
• Geode API for these data structures?
Contribution Ideas
22
![Page 23: Redis adaptor for Apache Geode](https://reader034.vdocuments.net/reader034/viewer/2022052405/58eed21d1a28ab13738b456b/html5/thumbnails/23.jpg)
Questions?
23