re-architecting with mongodb 

RE-ARCHITECTING WITH MONGODB 

FIXING A RELATIONAL MESS WITH A NON RELATIONAL DATABASE

ABOUT US

MICHAEL LUJANVP OF ENGINEERING @ VIXLET

BOBBY LINCOLNCOO @ VIXLET

A LITTLE ABOUT VIXLET

• LOCATED IN DOWNTOWN LA• A PASSION BASED SOCIAL NETWORK • SOCIAL NETWORKS FOR MLB, ATP,

LFC, SLIPKNOT, ETC

CAPSULES

• CAPSULES ARE USED TO REPRESENT TOPICS

• USERS AND BRANDS CAN POST INTO THESE CAPSULES

THE PULSE• THE PULSE IS WHERE YOU FIND

CONTENT FROM CAPSULES YOU HAVE SUBSCRIBED TO• SHOWS CAPSULE WHERE MEDIA IS

FROM• USER WHO POSTED CONTENT• WHETHER YOU HAVE LIKED THIS

CONTENT

MEDIA RECORDS

• ORIGINALLY DESIGNED BY DEVELOPERS WITH RDBMS EXPERIENCE

• FOREIGN KEYS JOINED AT RUNTIME IN CODE

{”_id”: ”

53d682bba374b38a686e9eeb”,“capsuleId”:

“53d681f9a374b38a686e9ee0”,“creator”:

“55b815ea54517d544e3f5bee”,“type”: “text”,“caption” “test”,“statuses”: {

“created”: 1445621494381,“updated“: 1465410655512

}}

ORIGINAL WORKFLOW

Feed MS Media MS Capsule MSoAuth MS

Capsule MS

Relationship MS

User MS

User MS

Relationship MS

AVERAGE RESPONSE TIMES

Call Response Time AveragesPulse 4000msCapsule Feed 2000msProfile Feed 2500ms

INSERT FUNNY PICTURE

TIME TO RE-ARCHITECT

• MICRO MODULE ARCHITECTURE• SINGLE API LAYER BUT BROKEN UP INTO BITE SIZE MODULES• REMOVES HTTP OVERHEAD OF SOME MICRO-SERVICE ARCHITECTURE• ALLOWS FOR SINGLE LAYER TO BE BROKE UP INTO MICRO-SERVICES• ALLOWS FOR EACH PIECE TO BE OPTIMIZED (INCLUDING DATABASES)

MEDIA CREATION

Service Layer

Create Relationship

sCreate Media

Record Add to feeds

Mongo Riak Riak

SIMPLIFY THE ARCHITECTURE

• WORKER ARCHITECTURE• CRUD IS HANDLED BY SERVICE LAYER• ALL NON CRITICAL WORK IS HANDED OFF TO WORKERS• MAIN DISPATCHER HANDLES ALL REQUESTS • IMPROVES SPEED AND SIMPLICITY

DISPATCHINATOR

• EVENT TRACKING• ERROR HANDLING / RETRY LOGIC

NEW MEDIA CREATION

Service Layer

Media Module

Dispatcher Queue

Create Relationship Neo4j

Mongo

CassandraAdd to Feeds

NEW MEDIA RECORDS

• COMPLETE CAPSULE AND CREATOR OBJECTS

• NO LONGER NEED TO JOIN ON FOREIGN KEYS

{ ”_id”: ” 53d682bba374b38a686e9eeb”, “capsule”: {

“id”: “53d681f9a374b38a686e9ee0”,“name”: “ATP Challenger Tour”,“cover”: “image.png”,“description”: “description”

}, “creator”: {

“id”: “55b815ea54517d544e3f5bee”,“username”: “atpworldtour”,“avatar”: “image.png”

}, “type”: “text”, “caption” “test”, “statuses”: {

“created”: 1445621494381,“updated“: 1465410655512

}}

CAPSULE / USER UPDATES

Service Layer Module Dispatcher

Queue

Update Media Mongo

Mongo

BUT THERE IS A PROBLEM

• WHAT HAPPENS WHEN THERE ARE 1,000 UPDATES? OR 1,000,000 UPDATES?

FAN OUT ON WRITE WORKER

• KEEP COUNT OF MEDIA RECORDS PER CAPSULE• BREAK UP WORK INTO MANAGEABLE SIZES

NEW UPDATE WORKFLOW

Service Layer Module Dispatcher

Queue

Fan Out Worker

MongoMongo Update

Media

NEW LOOK UP WORKFLOW

Service Layer

Feed Module

Relationship Module

NEW RESPONSE TIMES

Call Response Time AveragesPulse 200msCapsule Feed 150msProfile Feed 150ms

ANY QUESTIONS???