i love apis 2015: apache usergrid web scale mobile apis with elastic search and cassandra

1

Apache Usergrid: Web Scale Mobile APIs with ElasticSearch and Cassandra !

Todd Nine Principal Architect

Jeffrey West BaaS Engineering

Coach

Agenda

2

1. Why do we need Usergrid? 2. What is Usergrid & How does it work? 3. How do we achieve web scale? 4. Learnings 5. Roadmap & Questions

©2015 Apigee. All Rights Reserved.

3

(m) obile B ackend a s a S ervice

Datastore

Push Notifications

Connections /Social

Location

Users

…and more

Application

Systems of Engagement

4

Building Systems of Engagement with Edge API BaaS

5

Systems of "Record

Apigee Edge: Intelligent API Platform

Systems of Engagement

App Developers

Database ERP CRM Data "Warehouse Data Lake

Partner Apps Consumer Apps Employee Apps Insights

6 ©2015 Apigee. All Rights Reserved.

Web Scale"10’s of thousands requests/sec"…across multiple data centers


Web Scale / MongoDB

Usergrid

8

• Apache Open Source Mobile Backend as a Service • REST API for easy interaction from any platform • SDKs available for iOS / Android / Javascript

9

The Usergrid Taxonomy

10

Organization Organization

Organization

Application

Devices Users Groups

Security Roles

Collection Collection

Entity {…} Entity

{…} Entity {…}

Application

Devices Users Groups

Security Roles

Entity {…} Entity

{…} Entity {…}

Collection Collection

Entity {…} Entity

{…} Entity {…}

But I just want persistence!


REST API GET / PUT / POST / DELETE

Scalable, Multi-tenant Persistence (Single Cluster)

What people do seeking ‘web scale’?


The Usergrid Stack

14

•  Canonical System of Record

•  Collections –  Unique Values / Primary Keys

•  Entities –  Schemaless design –  O(1) entity retrieval

•  Connections / Graph –  Scalable Relationships

Horizontally Scalable Persistence

Built on Horizontally Scalable Technologies

• Filter / Limit / Cursor: – GET http://api.usergrid.com/{org}/{app}/{collection}?ql=select * where title = 'A Tale of Two Cities’&limit=50&cursor=LTIxNDg0NDUxNDpnR2tBQVFFQWdITUFDWFJ2YlM1emJXbDBhQUNBZFFBUUQyMVZneExfRWVLRlV3TG9Hc1doZXdDQWRRQVFIYVdjb0JwREVlS1VCd0xvR3NWT0JRQQ

– GET http://api.usergrid.com/{org}/{app}/{collection}?ql=select * where location within {d} of {x, y}


( )

Usergrid Application


Architecture: Multi-Region Indexing & Consistency

SQS SQS SNS

6) INDEX 6) INDEX

Region B Region A C* Replication

2) WRITE 5)

REA

D

5) R

EAD

Web Scale

20

Bottlenecks


Tomcat 20% CPU !?? WHY ??! Cassandra

10% CPU

Iterative Process


10,266/s 9,700/s

7,233/s

3,750

Hurdles •  Gatling

–  Can’t get 10k/s with one node •  Cassandra

–  CPU •  Compression? •  C3.2xlarge -> c3.4xlarge •  Row caching?

•  Tomcat –  HTTP Connector

•  acceptCount •  maxConnections •  maxThreads •  Non-blocking IO?

•  Usergrid –  # of reads to Cassandra


Scaling Usergrid to 10,000 Req/s w/ Gatling

•  10k test run: –  35 c3.xlarge Tomcat Nodes –  9 c3.4xlarge C* nodes –  6 m3.large ES nodes

•  Closing test ~7,800/s: –  15 c3.xlarge Tomcat Nodes –  9 c3.4xlarge C* nodes –  6 m3.large ES nodes

24

Elasticsearch(working… not so well…)

25

Learn from our mistakes

26

Risks of Dynamic Mapping


D

D

D

D M



DD

D

DD

DD

DDD

D D

DD

DD

DD

DD

Inbound message requires a mapping

update

Mapping updated & broadcast to the

cluster M



M

DD

D

DD

DD

DDD

D D

DD

DD

DD

DD

At scale (full loaded) only one mapping

update can processed at a time



DD

D

DD

DD

DDD

D D

DD

DD

DD

DD

M



M

DD

D

DD

DD

DDD

D D

DD

DD

DD

DD

30 ES Data Nodes X

100MB mapping file =

3,000MB

For each schema/mapping update

3GB data transfer X

5M entities with different schemas

= a LOT of data

transfer and a lot of latency



DD

D

DD

DD

DDD

D D

DD

DD

DD

DD

M

Elasticsearch(working well)

33

{ "hello": { "world":{ "message": "Hello!" }, "items": [ { "name": "apples", "weight": 100 } ] } }

{ "uuid": "48e262d6-b922-11e4-8a11", "ud_entity_id": "ad2b33c2e", "size": 100, "ug_app": "1a67aed7-bbb3-11e4-8a11", "fields": [ { ”name": "hello.world.message", ”string": "Hello!" }, { ”name": "items[0].name", ”string": "apples" }, { ”name": "items[0].weight", "integer": 100 } ] }

Flattened Documents


Indexed as

Data Node A

Data Node B

Data Node C

Index Allocation •  Multiple Indexes spread across the cluster •  Indexes have shards •  Read/Write aliases point to physical

indexes •  Aliases assigned based on hashing of the

App UUID


Index_1

1

1

2

3 2

3

app_1_read_alias app_1_write_alias

Data Node A

Data Node B

Data Node C

Tenant Management


Index_1

1

1

2

3 2

3

app_1_read_alias

app_1_write_alias

app_2_read_alias

app_2_write_alias

Data Node A

Data Node B

Data Node C

Tenant Management •  Assign Write alias to new Index •  Add new index to read alias •  Stream data from old index •  Delete data from old index


Index_1

1

1

2

3 2

3

Index_2

1

1

2

3

2 3

app_2_read_alias

app_2_write_alias

stream2es

app_1_read_alias

app_1_write_alias

Data Node D

Data Node A

Data Node B

Data Node C

Tenant Management •  Indexes targeted to separate machines


Index_1

1

1

2

3 2

3

Index_2

1

1

2

3 2

3

app_1_read_alias app_1_write_alias app_2_read_alias

app_2_write_alias

Data Node A

Data Node B

Data Node C

Data Node A

Data Node B

Data Node C

Multiple Scaling Vectors


Index_1

1

1

2

3 2

3

Index_2

1

1

2

3 2

3

Index_3

1

1 2

3

2 3

Index_4

1

1 2

3 2

3

Index_5

1

1

2

3 2

3

Working with Graph & NoSQL Data

40

Data De-normalization per interaction

41

ERP Product CatalogsCRM Data LakeData

Warehouse

Graph Data Persistence: Entities & Connections

•  Entities – Objects / Documents representing an Entity

•  Collections –  Logical grouping of Entities

•  Connections / Entity Relationships – Graph Data Structure –  /{collection}/{entityA}/{verb}/{collection}/

{entityB} –  /Users/me/likes/Products/iPhone6 –  /Recipes/Bread/includes/Ingredients/Flour

42

User A

Product A

User B

User C

Likes>> <<Liking

Roadmap

43

What’s Next for Usergrid

• Grow the community! • Blueprints Driver Release! • Service abstraction •  Indexing closer to real-time • Larger Clusters • Distributed Work System • Encryption & Security

44

http://usergrid.apache.org

45

i love apis 2015: apache usergrid web scale mobile apis with elastic search and cassandra

Software