temporal graph analytics with gradoop - fosdem.org · temporal graph analytics with gradoop...
TRANSCRIPT
![Page 1: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/1.jpg)
Temporal Graph Analytics with GRADOOP
Christopher Rost and Kevin GomezLeipzig University
![Page 2: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/2.jpg)
About the speakers and the team
Kevin, PhD Student
Christopher, PhD Student
Prof. Dr. Erhard Rahm,Database Chair
Timo, M.Sc. Student
Lucas, B.Sc. Student
![Page 3: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/3.jpg)
We’re hiring!
CENTER FORSCALABLE DATA ANALYTICSAND ARTIFICIAL INTELLIGENCE
● Visit us at www.scads.de
● Open positions and projects (Developer and PhD Students)
● Various topics:● Machine Learning (Graph-ML, Privacy)● Image Recognition● Blockchain (Distributed-Ledger)● Graph Analytics
![Page 4: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/4.jpg)
Motivation
![Page 5: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/5.jpg)
“Graphs are everywhere”
![Page 6: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/6.jpg)
“Graphs are everywhere”
![Page 7: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/7.jpg)
“Graphs are everywhere”
friendOf
friendOf
friendOf
friendOf
friendOf
friendOf
friendOf
frien
dOf
friendO
f
friendO
f
friendOf
friendOf
friendO
f
Anna
Bob
Chris
Sally
Dutch
Arthur
Jack
SheldonRaj
Mike
James
John
![Page 8: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/8.jpg)
“Graphs are everywhere”
friendOf
friendOf
friendOf
friendOf
friendOf
friendOf
friendOf
frien
dOf
friendO
f
friendO
f
friendOf
friendOf
friendO
f
Anna
Bob
Chris
Sally
Dutch
Arthur
Jack
SheldonRaj
Mike
James
Johnage:29sex: f
yob: 1991
![Page 9: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/9.jpg)
“Graphs are everywhere”
friendOf
friendOf
friendOf
friendOf
friendOf
friendOf
friendOf
frien
dOf
friendO
f
friendO
f
friendOf
friendOf
friendO
f
Anna
Bob
Chris
Sally
Dutch
Arthur
Jack
SheldonRaj
Mike
James
Johnage:29sex: f
yob: 1991
since: 5Y
![Page 10: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/10.jpg)
“A open-source framework and research platform for efficient, distributed and domain independent management and analytics of heterogeneous and
temporal graph data.“
![Page 11: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/11.jpg)
Architecture
HDFS / YARN
Distributed Operator Implementation
TPGMEPGM
G raph A nalytical L anguage (GrALa) Java 8
45k LOC
ALv 2.0
![Page 12: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/12.jpg)
Graphs and collections of graphs
![Page 13: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/13.jpg)
OperatorsOperators
Unary Binary
Log
ical G
rap
hG
rap
h C
ollecti
on
Algorithms
Frequent Subgraph
![Page 14: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/14.jpg)
Temporal extension
![Page 15: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/15.jpg)
Temporal extension
![Page 16: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/16.jpg)
Temporal extension
![Page 17: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/17.jpg)
Temporal extension
![Page 18: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/18.jpg)
Time dependent operatorsOperators
Unary Binary
Log
ical G
rap
hG
rap
h C
ollecti
on
Algorithms
Frequent Subgraph
![Page 19: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/19.jpg)
Time dependent operatorsOperators
Unary Binary
(Tem
p-)
Log
ical G
rap
h(T
em
p-)
Gra
ph
Collecti
on
Algorithms
Frequent Subgraph
![Page 20: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/20.jpg)
Virus XSymptoms: 3rd eye growing
Transmission: if contact > 5 minIncubation period: 5 time units
Surgery
Accident
Cardiology
Oncology
Contact trackingSensors capturing who is close to whom
at what time.{empID1, empID2, t_from, t_to}
In case of an infection, which
hospital services are at risk of contracting
the virus X?
Use case: spread of airborne pathogens
Images by Vecteezy.com.
![Page 21: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/21.jpg)
timet0 t1
t5 ... t
10 t
15 t
20
TemporalGraph contacts = mySource.getTemporalGraph();
TemporalGraph contacts = mySource.getTemporalGraph();
![Page 22: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/22.jpg)
time
Employee[-,-)id: 32432srv: Oncology
Employee[-,-)id: 99934srv: Oncology
contact[ t
2 ,t
6 )
t0 t1
t5 ... t
10 t
15 t
20
![Page 23: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/23.jpg)
timet0 t1
t5 ... t
10 t
15 t
20
Breaking news:
Employee of oncology infected by VirusX.
Detected at t20
.
![Page 24: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/24.jpg)
timet0 t1
t5 ... t
10 t
15 t
20
Breaking news:
Employee of oncology infected by VirusX.
Detected at t20
.
Virus XSymptoms: 3rd eye growing
Transmission: if contact > 5 minIncubation period: 5 time units
![Page 25: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/25.jpg)
timet0 t1
t5 ... t
10 t
15 t
20
Virus XSymptoms: 3rd eye growing
Transmission: if contact > 5 minIncubation period: 5 time units
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
![Page 26: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/26.jpg)
timet0 t1
t5 ... t
10 t
15 t
20
Virus XSymptoms: 3rd eye growing
Transmission: if contact > 5 minIncubation period: 5 time units
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
![Page 27: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/27.jpg)
timet0 t1
t5 ... t
10 t
15 t
20
Virus XSymptoms: 3rd eye growing
Transmission: if contact > 5 minIncubation period: 5 time units
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);
![Page 28: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/28.jpg)
timet0 t1
t5 ... t
10 t
15 t
20
Virus XSymptoms: 3rd eye growing
Transmission: if contact > 5 minIncubation period: 5 time units
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);
contactmaxDur: 40 min
contactmaxDur: 122 min
contactmaxDur: 124 min
contactmaxDur: 146 min
contactmaxDur: 2 min
contactmaxDur: 9 min
![Page 29: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/29.jpg)
timet0 t1
t5 ... t
10 t
15 t
20
Virus XSymptoms: 3rd eye growing
Transmission: if contact > 5 minIncubation period: 5 time units
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);
contactmaxDur: 40 min
contactmaxDur: 122 min
contactmaxDur: 124 min
contactmaxDur: 146 min
contactmaxDur: 2 min
contactmaxDur: 9 min
![Page 30: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/30.jpg)
timet0 t1
t5 ... t
10 t
15 t
20
contactmaxDur: 40 min
contactmaxDur: 122 min
contactmaxDur: 124 min
contactmaxDur: 146 min
contactmaxDur: 2 min
contactmaxDur: 9 min
Breaking news:
Employees of oncology and surgery quarantined
because of VirusX.
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);
![Page 31: Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University. About the speakers and the team Kevin,](https://reader034.vdocuments.net/reader034/viewer/2022052005/60197b5bd84fcf33bc1ecb6d/html5/thumbnails/31.jpg)
Conclusion● Distributed graph analysis system● Temporal property graph model
– Bitemporal support– Logical graphs and graph collections– Composing operators and algorithms
● Declarative workflow creation
● Visit Gradoop: http://gradoop.com● Read our Wiki: https://github.com/dbs-leipzig/gradoop/wiki● Start: https://github.com/dbs-leipzig/gradoop/wiki/Getting-started● Try the examples: https://github.com/dbs-leipzig/gradoop/wiki/Examples