nosql landscape and a solution to polyglot persistence- impetus webcast

8/12/2019 NoSQL Landscape and a Solution to Polyglot Persistence- Impetus Webcast

1/31

NoSQL Landscape and aSolution to Polyglot

Persistence


2/31

Impetus Technologies

Agenda

Big Data Problems

Transition from RDMS to NoSQL

NoSQL Landscape

Challenges in transition

Tools for NoSQL

Kunderaan open source polyglot solution

Recorded version available at http://bit.ly/1hfz4Tn Impetus Technologies
http://bit.ly/1hfz4Tnhttp://bit.ly/1hfz4Tn


3/31


BIG Data Problem

Recorded version available at http://bit.ly/1hfz4Tn


4/31


Why not RDBMS?

Scalability

Data format

High availability Data volume in

zeta byte,

yottabyte

Horizontal scaling

would be

expensive

Data format can be

static or dynamic

Relational / Non-

relational

Data locality

No single

point of failure



5/31


Non-RDBMS way

Scale out

Scale up Static schema

Dynamic schema

Centralized

Decentralized



6/31


Introduction to NoSQL

An approach to storing and retrieving data with horizontal

scaling, simple design and high availability

Data format

driven processing

Distributed with No

single point of

failure(SPOF)

Thinking out of

SQL box



7/31 Impetus Technologies

NoSQL : A Pragmatic Solution?

With NOSQL data can be consistent, highly

available and with no SPOF!

But not 100%!



8/31


CAP Theorem

Consistency

AvailabilityPartitionTolerance

N/A



9/31


Thinking NoSQL?

Size

Format

VelocityFiltering

Large

Data



10/31


Size

High data growth! scalability is an issue?

Traditional RDBMS based solutions will not work!

xxxxxxxxx



11/31


Velocity

Near real time/Big Data analytics

Parallel processing, ready-for-read design is required

Traditional RDBMS solutions are not

fast enough to meet the SLAs !



12/31


Filtering

Filtering. Fraud detection

Risk management analysis

Traditional RDBMS may work on small

scale but not with large data !



13/31


Format

Non relational data format.

Different nature of data set: graph based, key-value based access

Traditional database is limited to static tables!

l

o

g

s



14/31


NoSQL Landscape

NOSQL



15/31


Transition to NoSQL

DatastoreSelection

API

exploration

LandscapeUnderstandin

g

Implementation



16/31


17/31


High Level APIs

Kundera

Kundera Kundera

Hector

Easy

Cassandra

Datastax java

driver

Astyanax

Morphia

Data Nucleus

Jongo

Spring data

Spring data

Neo4j

Hibernate

OGM

Data nucleus

Hbase api

Spring data

Kundera



18/31


Hybrid Design

Online data

streaming

Cassandra, HBase

User

information

RDBMS

In memory

cache/session

Redis

Website

advertisement

MongoDB, Couchbase

User activity

graph

Neo4J, Titan

Batch processing

& analytics

Hadoop, Spark



19/31


Bumpy Ride!

Unlearn and Learn

new APIs!

Index based retrieval

over multiple NOSQL

data stores

Atomic operations

NOSQL world is still

evolving, may need to

explore among data

stores

Migration of existing

production applications

and many more



20/31


One Stop Solution

Master key, possible?Lets explore!



21/31


Polyglot Way

Migrating

existing

solutions

Guarantee

atomicity

Switch

databases



22/31


High Level Polyglot API

Spring data

Kundera

Spring data

Kundera

Spring data

Kundera

Spring data

Kundera

Lets implement in JPA way!



23/31


Kundera to the Rescue!!

Supports 8 data stores

Cassandra, Hbase,

MongoDB, Redis,

Neo4j, Oracle NoSQL,

CouchDB and any

RDBMS

CRUD / Strong Query

SupportObject Relationships

Handling

Datastore-Optimized

Persistence and Query

Approach

Interceptors / Events /

Caching

Connection Pool /

Fallback (Lucene)

Indexing Flexibility



24/31


Getting Started

Repositories

sonatype-nexus

Kundera Public Repository

https://oss.sonatype.org/content/repositories/releases

true

false

Maven Dependencies

com.impetus.kundera.client

kundera-cassandra

2.11

com.impetus. kundera.client

kundera-rdbms

2.11

Building From Sourcegit clone [email protected]:impetus-opensource/Kundera.git

mvn clean install

mailto:[email protected]:impetus-opensource/Kundera.githttp://bit.ly/1hfz4Tnhttp://bit.ly/1hfz4Tnmailto:[email protected]:impetus-opensource/Kundera.gitmailto:[email protected]:impetus-opensource/Kundera.gitmailto:[email protected]:impetus-opensource/Kundera.gitmailto:[email protected]:impetus-opensource/Kundera.gitmailto:[email protected]:impetus-opensource/Kundera.git


25/31


User Logs Sample App

@Entity

@Table(name = "user)

@IndexCollection(columns = { @Index(name = "emailId") })public class User {

@Id

@Column(name = "user_id")

private String userId;

@Column(name = "first_name")

private String firstName;

@Column(name = "last_name")

private String lastName;

@Column(name = "emailId")

private String emailId;

@OneToMany(cascade = CascadeType.ALL, fetch =

FetchType.LAZY)

@JoinColumn(name = "user_id")

private Set logs;

@Embedded

private PersonalDetail personalDetail;

public User() {

// Default constructor.

}

//Setters and Getters

@Entity

@Table(name = logs)

@Index(columns = { "body", created_at" }, index = true)public class UserLogs {

@Id

@Column(name = log_id")

private String logId;

@Column(name = "body")

private String body;

@Column(name = created_at")

@Temporal(TemporalType.DATE)

private Date createdDate;

public UserLogs() {

// Default constructor.

}

// Setters and Getters

User Entity UserLogs Entity



26/31


User Logs Sample App

Configuration : Persistence.xml


27/31


Switching Data stores

//create entity manager factory.

EntityManagerFactory emf = Persistence.createEntityManagerFactory(logCassandra,logMongo, properties);

EntityManager em = emf.createEntityManager();

..

em.persist(user);

Configuration : Persistence.xml

Persist Data



28/31


Performance & Benchmarks



29/31


Technical Challenges Addressed!

Distributed indexing over multiple NOSQL database e.g. Solr,

Elastic search

Plugin Kundera powered ES or Lucene indexer

Build your own library and simply plugin

Unlearn and Learn new APIs! Based on most popular JPA 2.0 specification

Atomicity guarantee and Transaction management

Built in support for JPA/JTA transaction and batch operations

NOSQL world is evolving, plan to switch databases?

Since its a JPA powered solution, reuse same code with almost no changes



30/31

Q A

Big Data Solutions and Services partner for Enterprises

[email protected]

mailto:[email protected]:[email protected]:[email protected]:[email protected]


31/31


Thank You!

Meet us at Hadoop Summit, San Jose

CIO Big Data Summit, Texas

Strata Conference + Hadoop World, New York

Gartner Symposium, Orlando

Try / Recommend Kundera

https://github.com/impetus-opensource/Kundera

@impetustech
https://github.com/impetus-opensource/Kunderahttps://github.com/impetus-opensource/Kunderahttps://github.com/impetus-opensource/Kunderahttps://github.com/impetus-opensource/Kundera

nosql landscape and a solution to polyglot persistence- impetus webcast

Documents