project 3 final - cmu 15-721 :: advanced database...
TRANSCRIPT
![Page 1: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/1.jpg)
GARBAGE COLLECTION
SAURABH KADEKODI RAJAT KATEJA TIANYUAN DING
![Page 2: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/2.jpg)
GOALS
PROBLEM STATEMENT
▸ To implement efficient garbage collection in Peloton
GOALS (REVISED)
▸ 75% - implement basic tuple recycling using vacuum
▸ 100% - implement epoch based co-operative gc
▸ 112.5% - lock-free implementations, GetMemoryFootprint()
▸ 125% - DDL garbage collection (deferred after discussion)
![Page 3: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/3.jpg)
METHODOLOGY
WITHOUT GC
Empty Tile Slot
Tile Group
TxnAdd new tuple
![Page 4: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/4.jpg)
METHODOLOGY
WITHOUT GC
Empty Tile Slot
Tile Group
TxnAdd new tuple
![Page 5: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/5.jpg)
METHODOLOGY
WITHOUT GC
Empty Tile Slot
Tile Group
TxnUpdate tuple 3
![Page 6: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/6.jpg)
METHODOLOGY
WITHOUT GC
Empty Tile Slot
Tile Group
Empty Tile Slot
TxnUpdate tuple 3
![Page 7: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/7.jpg)
METHODOLOGY
WITHOUT GC
Empty Tile Slot
Tile Group
Empty Tile Slot
Cannot mark this slot free yet (MVCC)Tuple Update
TxnUpdate tuple 3
![Page 8: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/8.jpg)
METHODOLOGY
WITHOUT GC
Empty Tile Slot
Tile Group
Empty Tile Slot
Can use this slot…
TxnUpdate tuple 3
![Page 9: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/9.jpg)
METHODOLOGY
WITHOUT GC
Empty Tile Slot
Tile Group
Empty Tile Slot
Can use this slot…
TxnUpdate tuple 1
![Page 10: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/10.jpg)
METHODOLOGY
WITHOUT GC
Empty Tile Slot
Tile Group
Empty Tile SlotEmpty Tile Slot
Can use this slot…
TxnUpdate tuple 1
![Page 11: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/11.jpg)
METHODOLOGY
WITHOUT GC
Empty Tile Slot
Tile Group
Empty Tile SlotEmpty Tile Slot
Can use this slot…
TxnUpdate tuple 1
![Page 12: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/12.jpg)
METHODOLOGY
WITHOUT GC
Empty Tile Slot
Tile Group
Empty Tile SlotEmpty Tile Slot
Can use this slot…
Unused Slots
TxnUpdate tuple 1
![Page 13: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/13.jpg)
METHODOLOGY
WITH GC
Empty Tile Slot
Tile Group
1
2
3
4
3
Actually Free
Possibly Free
1
![Page 14: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/14.jpg)
METHODOLOGY
WITH GC
Empty Tile Slot
Tile Group
1
2
3
4
3
Actually Free
Possibly Free
1
![Page 15: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/15.jpg)
METHODOLOGY
WITH GC
Empty Tile Slot
Tile Group
Empty Tile Slot
Add to possibly_free_list
1
2
3
4
5
3
Actually Free
Possibly Free
1
![Page 16: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/16.jpg)
METHODOLOGY
WITH GC
Empty Tile Slot
Tile Group
Empty Tile Slot
Move to actually_free_list
1
2
3
4
5
3 Actually Free
Possibly Free
1
![Page 17: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/17.jpg)
METHODOLOGY
WITH GC
Empty Tile Slot
Tile Group
Empty Tile SlotEmpty Tile Slot
Move to actually_free_list
1
2
3
4
5
6
3 Actually Free
Possibly Free
1
![Page 18: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/18.jpg)
METHODOLOGY
WITH GC
Empty Tile Slot
Tile Group
Empty Tile SlotEmpty Tile Slot
1
2
3
4
5
6
Actually Free
Possibly Free
1
Use recycled tuple slot
![Page 19: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/19.jpg)
METHODOLOGY
WITH GC
Empty Tile Slot
Tile Group
Empty Tile SlotEmpty Tile Slot
1
2
3
4
5
6
Actually Free
Possibly Free1
Use recycled tuple slot
![Page 20: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/20.jpg)
METHODOLOGY
WITH GC
Empty Tile Slot
Tile Group
Empty Tile SlotEmpty Tile Slot
1
2
3
4
5
6
Actually Free
Possibly Free
1
Use recycled tuple slot
![Page 21: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/21.jpg)
METHODOLOGY
MEMORY FOOTPRINT
Tile Group
Empty Tile SlotEmpty Tile Slot
1
2
3
4
5
6
3
Actually Free
Possibly Free
1
1
{-
![Page 22: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/22.jpg)
CONTRIBUTIONS
GC MODES
▸ Off (default)
▸ Vacuum
▸ Naïve co-operative
▸ Epoch based co-operative
WORKLOAD
▸ YCSB - 80% updates, 20% reads, 10 terminals, 100000 tuples and 100 sec runtime
▸ YCSB - 50% updates, 50% inserts, 10 terminals, 100000 tuples and 100 sec runtime
![Page 23: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/23.jpg)
CONTRIBUTIONS
VACUUM
▸ Separate thread started as Peloton bootstraps
▸ Periodically moves elements from possibly_free_list to actually_free_list
VACUUM TRADEOFFS
▸ Vacuum thread period vs GC efficiency
▸ # Recycled per vacuum invocation vs GC efficiency
![Page 24: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/24.jpg)
CONTRIBUTIONS
VACUUM TRADEOFFS MEASURED
Mem
ory
Usag
e (M
B)
8
8.75
9.5
10.25
11
Thro
ughp
ut (r
ps)
0
3250
6500
9750
13000
Vacuum Configurations (V-<sleep>-<# recycled>)
V-1-1000 V-2-1000 V-5-1000 V-5-2000 V-5-3000 V-5-5000
Throughput (rps)Memory Usage (MB)
![Page 25: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/25.jpg)
CONTRIBUTIONS
NAÏVE CO-OPERATIVE (COOPERATIVE)
![Page 26: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/26.jpg)
CONTRIBUTIONS
NAÏVE CO-OPERATIVE (COOPERATIVE)
PERFORM GC
![Page 27: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/27.jpg)
CONTRIBUTIONS
NAÏVE CO-OPERATIVE (COOPERATIVE)
PERFORM GCPER
![Page 28: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/28.jpg)
CONTRIBUTIONS
COOPERATIVE TRADEOFFS MEASURED
Mem
ory
Usag
e (M
B)
0
2.25
4.5
6.75
9
Thro
ughp
ut (r
ps)
0
7500
15000
22500
30000
Naïve Configurations (N-<# recycled>)
N-1 N-10 N-25 N-50 N-100 N-1000 N-5000
Throughput (rps)Memory Usage (MB)
![Page 29: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/29.jpg)
CONTRIBUTIONS
EPOCH CO-OPERATIVE (EPOCH)
ejoin
![Page 30: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/30.jpg)
CONTRIBUTIONS
EPOCH CO-OPERATIVE (EPOCH)
ejoin
![Page 31: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/31.jpg)
CONTRIBUTIONS
EPOCH CO-OPERATIVE (EPOCH)
ejoin
![Page 32: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/32.jpg)
CONTRIBUTIONS
EPOCH CO-OPERATIVE (EPOCH)
e
join
![Page 33: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/33.jpg)
CONTRIBUTIONS
EPOCH CO-OPERATIVE (EPOCH)
e
join
![Page 34: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/34.jpg)
CONTRIBUTIONS
EPOCH CO-OPERATIVE (EPOCH)
e
join
![Page 35: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/35.jpg)
ee
CONTRIBUTIONS
EPOCH CO-OPERATIVE (EPOCH)
e
join
leave
![Page 36: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/36.jpg)
e e
CONTRIBUTIONS
EPOCH CO-OPERATIVE (EPOCH)
e
join
leave
![Page 37: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/37.jpg)
e e
CONTRIBUTIONS
EPOCH CO-OPERATIVE (EPOCH)
PERFORM GCPER
e
FOR
join
leave
![Page 38: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/38.jpg)
CONTRIBUTIONS
EPOCH TRADEOFFS MEASURED
Mem
ory
Usag
e (M
B)
0
3.5
7
10.5
14
Thro
ughp
ut (r
ps)
0
1750
3500
5250
7000
Epoch Configurations (E-<# epochs>)E-1 E-5 E-10 E-INF
Throughput (rps)Memory Usage (MB)
![Page 39: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/39.jpg)
CONTRIBUTIONS
OVERALL GC COMPARISON
Mem
ory
Usag
e (M
B)
0
5
10
15
20
Thro
ughp
ut (r
ps)
0
7500
15000
22500
30000
Best ConfigurationsOFF VACUUM NAÏVE EPOCH
Throughput (rps)Memory Usage (MB)
![Page 40: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/40.jpg)
CONTRIBUTIONS
CACHE MISSES MATTER AND THEY DON'T!Ca
che
Miss
es
0
4.5
9
13.5
18
Best ConfigurationsOFF VACUUM NAÏVE EPOCH
Cache Misses (%)
![Page 41: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/41.jpg)
PROBLEMS
ENTIRE TABLE TRUNCATIONS
▸ Possibly_free_list may potentially end up with an absurdly large number of free slots
▸ Has to be handled as a special case
RECYCLING ACROSS TILE GROUPS
▸ Current recycling is at table granularity - i.e. across tile groups
▸ Different tile group schemas in the same table may become problematic
▸ Tradeoff: recycling granularity vs # tuples recycled
![Page 42: project 3 final - CMU 15-721 :: Advanced Database …15721.courses.cs.cmu.edu/spring2016/files/project3...GOALS PROBLEM STATEMENT To implement efficient garbage collection in Peloton](https://reader031.vdocuments.net/reader031/viewer/2022030700/5aeb3e407f8b9a585f8d5ff2/html5/thumbnails/42.jpg)
QUESTIONS?
Thank You…