distributed computing with hazelcast - brazil tour
TRANSCRIPT
www.hazelcast.com@noctarius2k
/** * Distributed Computing * with Hazelcast */
www.hazelcast.com@noctarius2k
Interactive!
www.hazelcast.com@noctarius2k
www.hazelcast.com@noctarius2k
Disclaimer# A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00002aaaaaf6d9c3, pid=2185, tid=1086892352 # # JRE version: 6.0_21-b06 # Java VM: Java HotSpot(TM) 64-Bit Server VM (17.0-b16 mixed mode linux-amd64 ) # # An error report file with more information is saved as: # /path/to/hs_err_pid2185.log # # This presentation may include certain forward-looking statements and projections provided by the Company. Any such # statements and projections reflect various estimates and assumptions by the Company concerning anticipated # results. These statements involve risks, uncertainties and assumptions and are based on the current estimates and # assumptions of the management of the Company as of the date of this presentation and are subject to uncertainty and # changes. No representations or warranties are made by the Company as to the accuracy of any such statements or # projections. Whether or not any such forward-looking statements or projections are in fact achieved will depend upon # future events some of which are not within the control of the Company. Accordingly, actual results may vary from the # projected results and such variations may be material. Statements contained herein describing documents and # agreements are summaries only and such summaries are qualified in their entirety by reference to such documents # and agreements.
www.hazelcast.com@noctarius2k
Who’s that dude?
• Chris(toph) Engelbert • Twitter fanatic: @noctarius2k • Weird 9+ Java-Years • Performance, GC, Fairytales • Apache Committer • Gaming, Travel Management, …
www.hazelcast.com@noctarius2k
• Chris(toph) Engelbert • Twitter fanatic: @noctarius2k • Weird 9+ Java-Years • Performance, GC, Fairytales • Apache Committer • Gaming, Travel Management, …
The www.hazelcast.ninja
Who’s that dude?
www.hazelcast.com@noctarius2k
DistributedComputing…
www.hazelcast.com@noctarius2k
ToasterServer 1
TV-Server 1Power
CoffeeMakerServer 1
Kettle Server 1
StoveServer 1
Power GridServer 1, 2
DistributedComputing…
www.hazelcast.com@noctarius2k
… but what meansDistributed Computing?OOO
( )
www.hazelcast.com@noctarius2k
www.hazelcast.com@noctarius2k
“In-Memory Computing”
www.hazelcast.com@noctarius2k
‘In-Memory Computing’
L1 Cache Reference L2 Cache Reference Main Memory Reference 1K send over 1 Gbps Network 4K read from SSD Seek on Spinning Disk Send Packet CA->AMS->CA
0.5 ns 7 ns
100 ns 10,000 ns
150,000 ns 10,000,000 ns
150,000,000 ns
0.01 ms 0.15 ms
10 ms 150 ms
www.hazelcast.com@noctarius2k
Evolution of:- Memory-Price- Memory-Size
www.hazelcast.com@noctarius2k
Data Distribution(Partitioning / Sharding)
www.hazelcast.com@noctarius2k
Data Distribution
String code = “JCP-TOUR-2015”
Map<String, TShirtDelivery> shirts = Maps.create()
shirts.put(code, new TShirtDelivery(“Your Address”))
www.hazelcast.com@noctarius2k
Data Distribution
String code = “JCP-TOUR-2015”
int hashcode = hash(code)
partitionId = hashcode % partitionCount
cluster.send(partitionId, code, delivery)
www.hazelcast.com@noctarius2k
Data Distribution
int hashcode = hash(code)
partitionId = hashcode % partitionCount
cluster.send(partitionId, code, delivery)
Must be a consistent hash function
Partition count must be constant
Can be calculated, smart routing possible
www.hazelcast.com@noctarius2k
Demo Time
www.hazelcast.com@noctarius2k
That was easy!But don’t tell anyone!
www.hazelcast.com@noctarius2k
Parallelize Processing
www.hazelcast.com@noctarius2k
Parallelize Processing
int[] values = [0..100000].map(randomInt())
int[][] blocks = split(values, 10000)
int[] subresult = blocks.foreach(_.sum())
int result = subresult.sum()
www.hazelcast.com@noctarius2k
Parallelize Processing
int[][] blocks = split(values, 10000)
int[] subresult = blocks.foreach(_.sum())
int result = subresult.sum()
No mutation, only new values
Independently computable
Might split further down (fork-join, work-stealing)
www.hazelcast.com@noctarius2k
Demo Time2
www.hazelcast.com@noctarius2k
I told you:Easy Peasy!
www.hazelcast.com@noctarius2k
Distributed Caching!
www.hazelcast.com@noctarius2k
Collect, store …
www.hazelcast.com@noctarius2k
…access fast when necessary!…access fast when necessary!
www.hazelcast.com@noctarius2k
Caches
www.hazelcast.com@noctarius2k
Evolution Of Caching
Custom
www.hazelcast.com@noctarius2k
http://gadizmo.com/roll-your-own-with-the-ham-dogger.php
Roll Your Own!Evolution Of Caching
www.hazelcast.com@noctarius2k
Caches Are Simple!public class Cache<K, V> { private final Map<K, V> cache = new CHM<>();
public void put(K key, V value) { cache.put(key, value); }
public V get(K key) { return cache.get(key); } }
www.hazelcast.com@noctarius2k
Even With TTL!
www.hazelcast.com@noctarius2k
And Auto-Cleanup
www.hazelcast.com@noctarius2k
What Is Wrong?
www.hazelcast.com@noctarius2k
What Is Wrong?
O(n)5 mio
Entries
www.hazelcast.com@noctarius2k
It was a trap!
www.hazelcast.com@noctarius2k
Evolution Of Caching
Custom
www.hazelcast.com@noctarius2k
Proprietary
Evolution Of Caching
Custom
www.hazelcast.com@noctarius2k
Proprietary
Evolution Of Caching
Open Source
Custom
www.hazelcast.com@noctarius2k
Proprietary
Evolution Of Caching
Open Source
Commercial
Custom
www.hazelcast.com@noctarius2k
Proprietary
Evolution Of Caching
Open Source
Commercial
Stupidly ExpensiveCustom
www.hazelcast.com@noctarius2k
http://technoodling.net/mili-power-crystal-battery-pack-review/
Terracotta /EHcache
OracleCoherence
Infinispan
JavaCollections
API
Hazelcast
JCache
Custom
www.hazelcast.com@noctarius2k
Success!Feels like
www.hazelcast.com@noctarius2k
Quick Shameless Plug :)Hazelcast!
www.hazelcast.com@noctarius2k
Hazelcast in 5 Facts• Java Collections API• Java Concurrency API
• Transparent Data Distribution• Drop-In Replacement
• Disruptively Simple(c) Rod Johnson (@springrod)
www.hazelcast.com@noctarius2k
Time to be Happy!
www.hazelcast.com@noctarius2k
Time to be Happy!
Apache License 2
www.hazelcast.com@noctarius2k
Time to be Happy!
www.hazelcast.org
www.hazelcast.com@noctarius2k
You can do it!
www.hazelcast.com@noctarius2k
Thank You!Any Questions?
@noctarius2k http://www.sourceprojects.org
http://github.com/noctarius
@hazelcast http://www.hazelcast.com http://www.hazelcast.org
http://github.com/hazelcast
www.hazelcast.com@noctarius2k
Thank You!Any Questions?
@noctarius2k http://www.sourceprojects.org
http://github.com/noctarius
@hazelcast http://www.hazelcast.com http://www.hazelcast.org
http://github.com/hazelcast
Free Hazelcast T-Shirthttp://bit.ly/tshirtcast
www.hazelcast.com@noctarius2k
Thank You!Any Questions?
@noctarius2k http://www.sourceprojects.org
http://github.com/noctarius
@hazelcast http://www.hazelcast.com http://www.hazelcast.org
http://github.com/hazelcast
Free Hazelcast T-Shirthttp://bit.ly/tshirtcast
We’re hiring!
www.hazelcast.com/careers