riak at kivra
DESCRIPTION
TRANSCRIPT
RIAK at KIVRABip Thelin@bipthelin
BIPCTO & Co-founder
OO / SQL-backgroundOpen Source / Tomcat Committer
Scalable platforms / HostingBeer Homebrewer / Judge
THELIN
API / Delivery serviceContent compositionContent creation Digital mailbox
ResponsiveWeb
iOS App
x
b
a
d
c
q
r
g
z
z
t u
i
e
e
p
y
f
State machineInternal, External & Time Events
Hard to test / debugError proneHardcoded flowNo overview
We outgrew
our system
QUOTE:
24 / 7 / 365
Distributable / ScALABLE
TESTABLE / OPERATIONAL
SECURE / Stable
Marriage is a wonderful institution, but who wants to live in an institution?
Our PhilosophyKeep it simple, StupidDo one thing and do it wellSeparation of concernRight tool for the jobHasty generalization
RIAK + ErlangLightweightDistributedScalableResilient to failure
Backend service
RIAK
REST API
Frontend Service
qz
x y
qz
x y
Internal ServicesExternal Services
Right datastorefor the job
3 x Riak Clusters
ONEBACKENDMany
(s)t o r u l e t h e m a l l
LeveldbBitcaskRiak-erlang-client (PB)JSON
RIAKstack
OAUTH2Authorization layer
on top of Riak
LAZYUPGRADE
-‐record(user, { password = <<"l0ln00b">, name = <<"noname">>, i_primary_email = <<"[email protected]">, i_primary_phone = <<"461337">>, i_ssn = <<"0123456789">>, l_tenant = {<<"B">>, <<"K">>}}).
Get object
Upgrade object
RIAK
Check version
[1...n]
LOGGING
1. MetricsAnything you'd want toplot on a graph
2. (Un)structuredloggingLog data
you'dwant toprocess
3. Applicationlogging Error,
Notice,Warning, etc
Lager Logglygithub.com/kivra/lager_loggly.git
Lager Sysloggithub.com/basho/lager_syslog
~197 SLOC ~23 SLOC
Lager{lager, [
{handlers, [ {lager_console_backend, info}, {lager_loggly_backend, [<<"Riak">>,
info, 5, 3, "https://<Loggly URI>"]},
{lager_loggly_backend, [<<"Riak">>, error, 5, 3, "https://<Loggly URI>"]},
{lager_file_backend, [ {"/path/error.log", error, 10485760, "$D0", 5}, {"/path/console.log", info, 10485760, "$D0", 5} ]}
]}, ...]},
DEPLOYMENTCentral configuration
Infrastructure as CodeBootstrap/Upgrade
Data modelingConflict resolution2i slowno M/R, Links, Search, etc..Key generationCaching
DESIGNDECISIONS
andTRADEOFFS
Riak CoreData encryptionSearchSession backend
What’s up?
