cwin16 - lyon - partner mark logic - the rise of nosql

THE RISE OF NOSQL

Mark van der Waals, Capgemini

Michel de Ru, MarkLogic

How NoSQL fits in your Enterprise Architecture

© COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

WHAT BUSINESSES WANTS

WHAT IT & ARCHITECTS NEED TO DELIVER

SLIDE: 2 © COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

Bring new products and services to market fast

Deliver a modern user experience

Interact effectively in a networked ecosystem

Improve operational efficiency

Manage business risk

Use data & intelligence as key asset & enabler

Stay compliant to regulation

Fast time to market

Agility in delivery

Flexibility to change

Reduced cost & complexity

Technological innovation

MOBILE FIRST NEW

RESEARCHREQUIREMENTS

Your business is changing… Your data & infrastructure are changing…

NEW APPS

EXPANSION SOCIAL MEDIA & COMMUNITIES


ANALYTICS

NEW MONETIZATION STRATEGIES

VIRTUALIZATION &CLOUD

NEW REGULATION

MERGERS & ACQUISITIONS

EMPLOYEE TURNOVER

INTERNET OF THINGS

MACHINELEARNING

� Separation of transactional & intelligence world

� Silo systems also in intelligence world

� Content Mgt & Knowledge Mgt

� Collaboration platforms

This proliferation of data is:

� Resulting in latency …no real time picture

� Not consistent & not governable & non-compliant

Challenges in Enterprise Architecture


� Collaboration platforms

� Archives (Rules on anonymizing & deleting data, Right to be forgotten)

� DWHs, Operational Data stores with predefinedschema’s

� Multiple copies of data & exponential growthFS data is replicated 5-12 times, only 10% of data is unique

� Not effective: Myth of One

� Not affordable

� Not agile or flexible

Root causes:1. Heterogenous data in silos2. Relational DB’s are not made to

solve this problem

As data structures get more complex and data volume grows, traditional relational databases—and their need for a pre-defined

schema are falling short.”

“schema are falling short.”

The Problem With the Relational Approach

1Take a Current State Snapshot

Design the New Data Model

Perform ETL

23

4


The Business Changes, The Requirements Change,The Source Data Changes

Create the Indexes

4

Build the Application

5

Restart Process6

OPERATIONAL DATA SOURCES BINARY TXT GEOJSONXML

JSON

� A Multi Model approach

� Decoupling of loading and using the data

� Load data & meta data AS IS (one schema on write)

� So no upfront modelling & ETL tools needed

� Allows for multiple schema’s on read – per use case

Why NoSQL is part of the solution


BIDIRECTIONAL ANALYSISOF ALL DATA

MULTI-CHANNEL DISTRIBUTION

XML

SEMANTIC DATA

GEOSPATIAL DATA

BINARY

� Allows for multiple schema’s on read – per use case

� Allows for Agile & Parallel way of developing use cases

� Combining structured and semi/unstructured data such as:

� Documents (XML, enriched...)

� Geospatial data (2D)

� JSON & Binary data (audio,video)

� Relating and structuring data in a flexible way

� key-value pairs, graphs

Scalability and Elasticity

Why MarkLogic: the only enterprise hardened NoSQL platform

Enterprise hardened� Security

� Transactional integrity


Scalability and Elasticity� Horizontal Scaling on Commodity Hardware

� Flexible clustering for elastic scaling of user queries, transactions, and data volumes

� Data can be easily moved, deleted, designated read-only, taken offline/online, etc.

Use Cases for NoSQL and MarkLogic references� Record keeping & regulatory reporting

� Regulatory reporting

� Combining Archiving & Intelligence

� Operational Data Warehouse

� Content re-purposing

� B2C publishing

� B2B publishing

� Custom reporting


� Operational Data Warehouse

� Operational decision making in real-time

� Fraud prevention

� Next best action

� Application processing in banking & insurance

� Derivatives core processing

� Custom reporting

� Video metadata delivery

NoSQL in Regulatory ReportingA real world use-case

Research /

Portal and Channels

Execution & Position Management

CFTCMIFID I/II

SEC-Reg-SCISEC Rule 15c3-5

Treasury(Funding and

Liquidity)

CRD-IV

Finance & Finance &

Risk Management

BASEL II/IIICOREP

Dodd-FrankSEC-PF

Solvency II

BCBS 239

The Regulatory Landscape


Client Onboarding

KYC

Research / Advisory

Finance & General Ledger

Sarbanes-Oxley

FINREPIFRS

Solvency II

Operations

EMIRFATCA

Corporate Finance /

M&A

Legal / Compliance / Surveillance

MADII

Reference DataMIFIR

The Endless Cycle of Data Harmonization

1 2Take snapshot of current data Build master data model based

on initial view


34 Extract, transform, &load data into data model

Revise static model &restart process for new data

x

The Endless Cycle of Data Harmonization

1 2Take snapshot of current data Build master data model based

on initial view


34 Extract, transform, and load data into data model

Revise static model and restart process for new data

x

2-5 years$5M++

Simple and Fast Data Integration With NoSQL

1 2Load data “as-is” - index data now and transform over time

Agile application development without constraints - andwith a stable data layer


Dat

a la

yer

Time-to-completion: 3 monthsTime-to-completion: 3 months

Where do we go from here ?

Operational Data Hub

Operational

360 of Anything

Is-AMay be

applied toIs-A

Provides

Governance Risk & Compliance (GRC)

� Operational Transaction Repository (formerly known as Trade Store) is a type of ODH

� Covers Orders, Trades and Settlement Payments (Money, Shares,


Operational Transaction Repository

Aggregated Liquidity & Risk Reporting (CRD IV, CCARS

Solvency II)

Regulatory Book of Record RBOR (Dodd Frank, MIFID

II)

Fraud / Market Abuse Surveillance

Consolidated Position and Portfolio Management

May be used for

Transaction 360

Provides

Industry Function

Use Case

Architecture Solution

Business Solution

Enterprise

FinServe

Trading Utilities/SDRs and Regulators

(Money, Shares, Commodity Assets)

Capgemini – MarkLogic partnership� Joint projects & bids

� Training program:

� 30 trained Capgemini consultants in India, France, Spain & the Netherlands


cwin16 - lyon - partner mark logic - the rise of nosql

Technology