nosql databases and polyglot applications

48
Bases de Datos NoSQL Y Aplicaciones Poliglotas Agus%n Magaña Falconi

Upload: agustin-magana

Post on 25-Jun-2015

531 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: NoSql Databases and Polyglot Applications

Bases  de  Datos  NoSQL  Y  Aplicaciones  Poliglotas  

Agus%n  Magaña  Falconi  

Page 2: NoSql Databases and Polyglot Applications
Page 3: NoSql Databases and Polyglot Applications

AGENDA  

•  Un  poco  de  historia  •  Definición  de  NoSQL  •  Tipos  de  NoSQL  •  ¿Porqué  Aplicaciones  Poliglotas?  •  Ejemplo  usando  REDIS  

Page 4: NoSql Databases and Polyglot Applications

Edgar  Frank  "Ted"  Codd  

Page 5: NoSql Databases and Polyglot Applications

1980’s  

Page 6: NoSql Databases and Polyglot Applications

Tendencia:  Menos  uniformidad  

Page 7: NoSql Databases and Polyglot Applications

Datos  dispersos:  BD  Relacionales  

Page 8: NoSql Databases and Polyglot Applications

Tendencia:  Crecimiento  exponencial  de  datos  

Page 9: NoSql Databases and Polyglot Applications

Not  Only  SQL    

Page 10: NoSql Databases and Polyglot Applications

Not  Only  SQL    

•  No  usan  SQL  como  lenguaje  principal  de  consultas  •  No  se  requieren  estructuras  fijas  como  tablas  •  No  JOINsa  •  No  se  garanAza  ACID  •  Escalan  bien  horizontalmente  

Page 11: NoSql Databases and Polyglot Applications

Tipos  de  Bases  de  Datos  NoSQL  

•  Key/Value  stores  •  Document  Databases  •  Graph  Databases  •  Column  Oriented  Databases  

Page 12: NoSql Databases and Polyglot Applications

Key/Value  Stores  

•  Apache  Cassandra  •  BigTable  de  Google  •  Dynamo  de  Amazon  •  Voldemort  de  LinkedIn  •  Memcached  •  Oracle  NoSQL  Database  •  Redis  

Page 13: NoSql Databases and Polyglot Applications

Graph  Databases  

•  Neo4j  •  DEX  •  AlegroGraph  •  OrientDB  •  InfiniteGraph  •  InfoGraph  •  HyperGraphDB  

Page 14: NoSql Databases and Polyglot Applications

Document  Databases  

•  MongoDB  •  CouchDB  •  BaseX  •  Djondb  •  SimpleDB  •  Terrastore  

Page 15: NoSql Databases and Polyglot Applications

Column  Oriented  Databases  

•  Cassandra  •  BigTable  •  Hbase  •  Hypertable  

Page 16: NoSql Databases and Polyglot Applications

¿Aplicaciones  Políglotas?  

Page 17: NoSql Databases and Polyglot Applications

Food To Go Architecture

Order taking

Restaurant Management

MySQL Database

CONSUMER RESTAURANT OWNER

Page 18: NoSql Databases and Polyglot Applications

Limitaciones  de  las  Bases  de  Datos  Relacionales  

•  Escalabilidad  •  Distribución  •  Modifición  del  schema  •  O/R  impedance  mismatch  •  Manejor  de  datos  semi-­‐estructurados      

Page 19: NoSql Databases and Polyglot Applications

Solución:  Gastar  mucho  dinero  

Page 20: NoSql Databases and Polyglot Applications

Solución:  Usar  NoSQL  

•  Alto  performance    •  Alta  escalabilidad  

•  Rico  modelo  de  datos  

•  Schema-­‐less  

Beneficios  •  Limitaciones  en  transacciones  

•  Limitaciones  en  queries    •  Consistencia  no  garanAzada  

•  Datos  sin  constrains  

Desventajas  

Page 21: NoSql Databases and Polyglot Applications

Redis  

•  Almacenamiento  key-­‐value  avanzado  

•  Muy  rápida  (100K  reqs/seg)  

•  Persistencia  opcional  

•  Transacciones  con  bloqueo  opAmista  

•  Replicación  de  información  Master/Slave  

Page 22: NoSql Databases and Polyglot Applications

Sorted sets

myset a

5.0

b

10.

Key Value

ScoreMembers are sorted by score

Page 23: NoSql Databases and Polyglot Applications

Adding members to a sorted setRedis Server

zadd myset 5.0 a myset a

5.0

Key Score Value

Page 24: NoSql Databases and Polyglot Applications

Adding members to a sorted setRedis Server

zadd myset 10.0 b myset a

5.0

b

10.

Page 25: NoSql Databases and Polyglot Applications

Adding members to a sorted setRedis Server

zadd myset 1.0 c myset a

5.0

b

10.

c

1.0

Page 26: NoSql Databases and Polyglot Applications

Retrieving members by index range

Redis Server

zrange myset 0 1

myset a

5.0

b

10.

c

1.0ac

Key StartIndex

EndIndex

Page 27: NoSql Databases and Polyglot Applications

Retrieving members by score

Redis Server

zrangebyscore myset 1 6

myset a

5.0

b

10.

c

1.0ac

Key Min value

Max value

Page 28: NoSql Databases and Polyglot Applications

Redis  es  bueno  pero  ]ene  desventajas  

•  Búsquedas  solo  por  PK  

•  Modelo  de  transacciones  limitado:  

•  Lee  primero  y  ejecuta  updates  como  batch.  

•   Los  datos  deben  de  caber  en  memoria  

•  Le  faltan  funcionalidad  de  controles  de  acceso    

Page 29: NoSql Databases and Polyglot Applications

Caching with Redis

Order taking

Restaurant Management

MySQL Database

CONSUMER RESTAURANT OWNER

RedisCacheFirst Second

Page 30: NoSql Databases and Polyglot Applications
Page 31: NoSql Databases and Polyglot Applications

TimeRange MenuItem

Domain object to key-value mapping?

Restaurant

TimeRange MenuItem

ServiceArea

K1 V1

K2 V2

... ...

Page 32: NoSql Databases and Polyglot Applications

Finding available restaurants

Available restaurants =Serve the zip code of the delivery address

AND Are open at the delivery time

public interface AvailableRestaurantRepository {

List<AvailableRestaurant> ! findAvailableRestaurants(Address deliveryAddress, Date deliveryTime); ...}

Page 33: NoSql Databases and Polyglot Applications

Food to Go – Domain model (partial)

class Restaurant { long id; String name; Set<String> serviceArea; Set<TimeRange> openingHours; List<MenuItem> menuItems;}

class MenuItem { String name; double price;}

class TimeRange { long id; int dayOfWeek; int openTime; int closeTime;}

Page 34: NoSql Databases and Polyglot Applications

Database schemaID Name …

1 Ajanta

2 Montclair Eggshop

Restaurant_id zipcode

1 94707

1 94619

2 94611

2 94619

Restaurant_id dayOfWeek openTime closeTime

1 Monday 1130 1430

1 Monday 1730 2130

2 Tuesday 1130 …

RESTAURANT table

RESTAURANT_ZIPCODE table

RESTAURANT_TIME_RANGE table

Page 35: NoSql Databases and Polyglot Applications

Finding available restaurants on Monday, 6.15pm for 94619 zipcode

Straightforward three-way join

select r.*from restaurant r inner join restaurant_time_range tr on r.id =tr.restaurant_id inner join restaurant_zipcode sa on r.id = sa.restaurant_id where ’94619’ = sa.zip_code and tr.day_of_week=’monday’ and tr.openingtime <= 1815 and 1815 <= tr.closingtime

Page 36: NoSql Databases and Polyglot Applications

BUT how to implement findAvailableRestaurants() with Redis?!

?select r.*from restaurant r inner join restaurant_time_range tr on r.id =tr.restaurant_id inner join restaurant_zipcode sa on r.id = sa.restaurant_id where ’94619’ = sa.zip_code and tr.day_of_week=’monday’ and tr.openingtime <= 1815 and 1815 <= tr.closingtime

K1 V1

K2 V2

... ...

Page 37: NoSql Databases and Polyglot Applications

Where we need to be

ZRANGEBYSCORE myset 1 6

select value,scorefrom sorted_setwhere key = ‘myset’ and score >= 1 and score <= 6

=

key value score

sorted_set

Page 38: NoSql Databases and Polyglot Applications

We need to denormalize

Think materialized view

Page 39: NoSql Databases and Polyglot Applications

Simplification #1: Denormalization

Restaurant_id Day_of_week Open_time Close_time Zip_code

1 Monday 1130 1430 94707

1 Monday 1130 1430 94619

1 Monday 1730 2130 94707

1 Monday 1730 2130 94619

2 Monday 0700 1430 94619

SELECT restaurant_id FROM time_range_zip_code WHERE day_of_week = ‘Monday’ AND zip_code = 94619 AND 1815 < close_time AND open_time < 1815

Simpler query: No joins Two = and two <

Page 40: NoSql Databases and Polyglot Applications

Simplification #2: Application filtering

SELECT restaurant_id, open_time FROM time_range_zip_code WHERE day_of_week = ‘Monday’ AND zip_code = 94619 AND 1815 < close_time AND open_time < 1815

Even simpler query• No joins• Two = and one <

Page 41: NoSql Databases and Polyglot Applications

Simplification #3: Eliminate multiple =’s with concatenation

SELECT restaurant_id, open_time FROM time_range_zip_code WHERE zip_code_day_of_week = ‘94619:Monday’ AND 1815 < close_time

Restaurant_id Zip_dow Open_time Close_time

1 94707:Monday 1130 1430

1 94619:Monday 1130 1430

1 94707:Monday 1730 2130

1 94619:Monday 1730 2130

2 94619:Monday 0700 1430

key

range

Page 42: NoSql Databases and Polyglot Applications

Simplification #4: Eliminate multiple RETURN VALUES with concatenation

SELECT open_time_restaurant_id, FROM time_range_zip_code WHERE zip_code_day_of_week = ‘94619:Monday’ AND 1815 < close_time

zip_dow open_time_restaurant_id close_time

94707:Monday 1130_1 1430

94619:Monday 1130_1 1430

94707:Monday 1730_1 2130

94619:Monday 1730_1 2130

94619:Monday 0700_2 1430

...

Page 43: NoSql Databases and Polyglot Applications

zip_dow open_time_restaurant_id close_time

94707:Monday 1130_1 1430

94619:Monday 1130_1 1430

94707:Monday 1730_1 2130

94619:Monday 1730_1 2130

94619:Monday 0700_2 1430

...

Sorted Set [ Entry:Score, …]

[1130_1:1430, 1730_1:2130]

[0700_2:1430, 1130_1:1430, 1730_1:2130]

Using a Redis sorted set as an index

Key

94619:Monday

94707:Monday

94619:Monday 0700_2 1430

Page 44: NoSql Databases and Polyglot Applications

Querying with ZRANGEBYSCORE

ZRANGEBYSCORE 94619:Monday 1815 2359

{1730_1}

1730 is before 1815 Ajanta is open

Delivery timeDelivery zip and day

Sorted Set [ Entry:Score, …]

[1130_1:1430, 1730_1:2130]

[0700_2:1430, 1130_1:1430, 1730_1:2130]

Key

94619:Monday

94707:Monday

Page 45: NoSql Databases and Polyglot Applications

Sorry  Ted    

Page 46: NoSql Databases and Polyglot Applications

The future is polyglot

IEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg

e.g. Netflix• RDBMS• SimpleDB• Cassandra• Hadoop/Hbase

Page 47: NoSql Databases and Polyglot Applications

Algunos  usarios  NoSQL  

Page 48: NoSql Databases and Polyglot Applications

Gracias