challenges operating and scaling graphenedb by francisco fernandez
TRANSCRIPT
Francisco Fernández - @[email protected]
Agenda● Introduction
● Technologies and architecture
● Scalability challenges
Introduction
101
Fran Albertoworks_with
I want a neo4j db https://neourl.graphenedb.com
Customers
2012
First proofs of concept around graph dbs.
02/2013
Beta, early access to sandbox databases
10/2013
First dedicated production instances
06/2014
GA of shared & dedicated production plans
Present
HA clusters, advanced performance metrics, ...
DistributedTeam
Agenda
● Almost totally Async
● RFCs driven development (almost like an open source project)
● Some challenges around communication.
Technologies
...
We embrace the actor model
Thinking like an Erlanger
Actor hierarchies
Root
Foo Bar
a b c
d e f g
h
root/foo/a/h
Supervision
Root
Foo Bar
a b c
d e f g
h Notify to parent on fail
FSMHeartbeat protocol
Wait Resp
AlarmOperational
ok
ok
err
Check serv
Erlang doc
dramatization
Architecture
API
Web
Orch
Conf Provider
Scheduler
Station
SQS
Neo4jEventConsumer
Backups
Overseer
MetricsLogstash
Bronze Age
Station
Neo4j
Container - c
Overseer
Iron Age
Station
Neo4j
Container
Overseer
Scalability challenges
Backups
Don’t test backups as you go
Consistency Check is mandatory
Some checks took many hours
min complexity O(|V| + |E|)
Bolt
< 3.x
< 3.x
HTTP only
+
>= 3.x
HTTP && Bolt
ELK
Station
MetricsProcesso
r
StationStation
SPOF
MetricsProcesso
r
Station
Distribute load
MetricsProcesso
r
StationStation
Questions?
We’re [email protected]