big data concepts and patterns - inria · thinking big big data concepts and patterns l . p dr....

Thinking BIGBig Data Concepts and

Patterns

l . p

Dr. Lilia SfaxiData Science Wokshop, ENIT

September, 13th 2017

DATA2

DATAData Mining

DataViz Data Analytics

Open Data

Data Science

Cloud Mobile IoT BI

Big Data

3

CollectionProcessing

Storage

Visualization

Security - Monitoring

Business

Data Issues

4

Data Issues

Available Data Value Extraction On-Time5

Data Issues

Infrastructure

Data Issues

Infrastructure

Scalable

Available

Flexible

Volume

Velocity

Variety 7

Data Issues

3V 8

Store THEN Process

Big Data Principles

Data-driven Decisions (NOT Decision-Driven Data)

Redunduncy is GOOD

There is NO NEEDLESS Data

GO Polyglot!9

ScalabilityScale UP vs Scale OUT

10

Classical : Scale UP

ScalabilityScale UP vs Scale OUT

11

Big Data : Scale OUT

Ring Master- Slave12

ScalabilityArchitectures

Classical Architectures Big Data Architecture

Database Application Server

13

ScalabilityCo-Localization of Processing & Storage

14

ScalabilityFault Tolerance

Data Replication

15

ScalabilityFault Tolerance

Data ReplicationCluster Replication

16

ScalabilityFault Tolerance

Data ReplicationCluster ReplicationRack Awareness

AvailabilityCAP Theorem

17

Consistency

Availability Partition Tolerance

Pick Any Two

AvailabilityCAP Theorem

18

Consistency

Atomicity

Isolation

Durability

Basically Available

Soft-State

Eventual Consistency

ACID BASE

AvailabilityTime

19

Stream Processing Support

Dynamic and Interactive Charts and Reports

In-Memory Processing

In-Memory Storage

FlexibilityONE application can support…

20

Diverse Data Sources

Schema-less Data

Multiple Processing Paradigms

Multiple Storage Systems

Research Domains

21

In the Big Data DomainEVERYTHING

Is Yet to Be Done

Research DomainsOptimization

22

Processing Time Optimization

Storage Size and Compression

Data Access Optimization

Tradeoff bw Consistency and Availability

Research DomainsData Science

23

Distributed Algorithms for Machine Learning

Semantic & Sentiment Analysis

Visualization Algorithms

Data Mining, Data Prediction, Data Analytics

Research DomainsBig Data Design

24

Design Methodologies for Big Data Systems

Standardization of Big Data Architectures

Design and Architectural Patterns

Modeling Language(s) for Schema-less Data

Research DomainsBig Data Security

25

Non-Relational Databases Security

Logs Gathering and Analysis

Source Data Validation and Filtering

Access Control and Cryptography

Research DomainsBig Data & OtherTrends

26

Big Data & Business Intelligence

Big Data & Cloud Computing

Big Data & Internet of Things

Big Data & Mobile

Research DomainsBig Data & Business Fields

27

Big Data in Education

Big Data in Health

Big Data in Art

Big Data in Finance

In Other Words…

28

Big Data is Here to STAY!

Thinking BIGBig Data Concepts and

Patterns

l . p

Dr. Lilia SfaxiApril 2017