bi saas & cloud strategies for telcos

Brussels 21 – 22

March ‘13

BI Software as a Service & Cloud Strategies for Telcos David M Walker Data Management & Warehousing

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

1

Brussels 21 – 22

March ‘13 Cloud Computing (Since 1970) In 1970 a service provider

would have owned a mainframe and provided clients with a connecFon that allowed them to

interact with the system. Many companies would have accessed this system sharing the resource on

demand The terminology has changed and the technology has been updated

but the funcFonality remains the same. “Cloud compuFng is the use of compuFng resources (hardware and soMware) that are delivered as a service over a network (typically the Internet). Cloud compuFng entrusts remote services with a user's

data, soMware and computaFon.” -‐ Wikipedia

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

2

Brussels 21 – 22

March ‘13 Cloud Computing Architecture

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

3

Brussels 21 – 22

March ‘13 The ‘as a Service’ Stack •  The more standard the business applicaFon the higher up the stack it is possible to go

•  Most organisaFons can do e-‐mail, ERP, CRM as SaaS

•  There is also NaaS – Network as a Service that provides the connecFvity to the cloud & between clouds

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

4

Brussels 21 – 22

March ‘13 Cloud Models •  Public •  Services provided by a service provider, usually offered on a pay per use model measured by CPU, Storage, Network Bandwidth, AcFve Seats, etc.

•  Private •  Services built by an organisaFon for it’s own use but using the technologies and techniques of public cloud compuFng – really only available to large corporates and governments

•  Community •  Services built by a group of organisaFons who use common resources or share common concerns (security, compliance, etc.)

•  Hybrid •  CombinaFon of Public and Private/Community environment using allowing commodity components where possible and bespoke cloud services where necessary

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

5

Brussels 21 – 22

March ‘13 Public vs. Private Cloud •  Public cloud has an iniFal (very) low cost with predictable long-‐term cost – although the long-‐term cost is oMen considered quite high – private cloud is costly to setup and long-‐term cost is unpredictable

•  Scale-‐up and scale-‐out are easier on public cloud as the service provider is responsible for ensuring resources are available at a pre-‐determined cost

•  Security & privacy of data on public cloud will always be a concern. There are addiFonal concerns over data locaFon – can data be moved between states? Can rouge states host their applicaFons in the same hosted environment as your company servers? etc.

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

6

Brussels 21 – 22

March ‘13 Elastic Services •  In order to get more resource cloud based services have ways to enable organisaFons to get more physical or virtual servers quickly and easily

•  ElasFc services allow organisaFons to expand or contract the size of their individual servers (CPU/Disk/Bandwidth) based on need •  This is not usually transparent as it oMen requires a reboot •  ElasFc disk space is usually slower than dedicated disks

•  Choice of database technology will affect choices •  Appliances need to be hosted or co-‐located •  Column-‐store databases are very effecFve in elasFc compute as the they use less resources compared to the raw data or row based dbs

•  Big Data technologies favour large numbers of elasFc compute technologies but this can get expensive

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

7

Brussels 21 – 22

March ‘13 The BI Systems Model

•  Most BI/DWH soluFons do & will look something like this •  There will be a corporate strategy for operaFonal systems •  Legacy systems will be in-‐house •  New systems may be in-‐house or in the cloud •  Even if an organisaFon is commifed to a fully cloud based future there will be a transiFonal state which is mixed mode

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

8

Source Systems

Data Warehouse Database

ReporFng Tools Servers

Data Mart Database ETL Server ETL

Server Staging

Database

Staging

Database

Brussels 21 – 22

March ‘13 Deployment Model 1

•  ExisFng Data Warehouse retained in house •  Data Marts & ReporFng Tools migrated to the cloud •  ETL Server has to deliver ‘ETL’ (i.e. data ready for direct load) •  ReporFng via web interface •  Desktop tools used via Remote Desktop

•  Favoured as a first step into the cloud •  Experiments in public cloud and for extranet usage

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

9

In House In The Cloud

Source Systems




Server Staging

Database

Staging

Database

Brussels 21 – 22


•  Deployment with two ETL servers •  First ETL Server loads the data warehouse •  Second ETL server has to deliver ‘ELT’ to ensure that performance is sufficient and uses a pull model

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

10


Source Systems




Server Staging

Database

Staging

Database

Brussels 21 – 22


•  Deployment with two ETL servers •  First ETL Server loads the staging area, this is normally a literal staging area and therefore is just ‘EL’ (no ‘T’)

•  Second ETL server has to deliver ‘ELT’ to ensure that performance is sufficient and uses a pull model

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

11


Source Systems




Server Staging

Database

Staging

Database

Brussels 21 – 22


•  Deployment with a single cloud based ETL server •  Normal BI operaFons but all of the BI is in the cloud

•  More oMen used with Private cloud due to security concerns

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

12


Source Systems




Server Staging

Database

Staging

Database

Brussels 21 – 22


•  Fully cloud based environment •  No issues around data movement volumes •  Security concerns will have been addressed because all corporate data is in the cloud already

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

13

In The Cloud

Source Systems




Server Staging

Database

Staging

Database

Brussels 21 – 22

March ‘13 Legacy Deployment Model

•  Fully cloud based environment for operaFonal systems •  Data being dragged back into legacy BI system •  Issues with migraFng legacy BI technologies to the cloud

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

14

In The Cloud

Source Systems




Server Staging

Database

Staging

Database

In House

Brussels 21 – 22

March ‘13 Major Issues •  Volume of data over the network •  Whichever (distributed) deployment model is used at some point significant amounts of data will have to moved across the network – this is the slowest component of any BI soluFon and introduces cost and latency

•  Punching a hole in the firewall •  Many organisaFons will be concerned about deploying an ETL tool in the cloud that needs fill read access to all it’s major systems – deployment models that restrict this access tend to be more popular

•  Compliance •  Can you move your data outside your state and any other range of regulaFons

•  US -‐ FISMA,HIPAA, SOX; EU – Data ProtecFon DirecFve; etc.

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

15

Brussels 21 – 22

March ‘13 Appliances in the cloud •  All physical appliances in the cloud have to be hosted or co-‐located •  Oracle Exadata, IBM Netezza, InfoBright Infopliance •  All offer some managed service, implementaFon with preferred partners or self-‐build cloud opFons

•  Virtual Appliance providers allow elasFc services to be used as well as being hosted or co-‐located •  EMC Greenplum, ParAccel •  These technologies have community ediFons which allow try before you buy

•  Again all offer some managed service, implementaFon with preferred partners or self-‐build cloud opFons

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

16

Brussels 21 – 22

March ‘13 Column Store In the Cloud •  Column Store databases well suited to cloud implementaFons •  UK operator data being loaded into a column store database •  2.6 Billion call data records (CDR) per day •  114Gb of data in raw files including reference data •  < 2.5 hours to transfer to server and load data •  Database grows by 7Gb per day – effecFve compression 16.25:1

•  Players in the market offering services in this area include: •  AcFan Vectorwise*, HP VerFca, InfoBright*, MicrosoM SQL Server 2012, SAP Sybase IQ

•  *These technologies have community ediFons which allow try before you buy – others have low cost PoC opFons

•  All available in co-‐locaFon, hosted and elasFc service models

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

17

Brussels 21 – 22

March ‘13 Big Data in the Cloud •  The technologies that was designed for use in the cloud from the beginning.

•  Lots of emerging technologies and companies •  Current ones which are definitely interesFng include Acunu, Cassandra, DataStax, Cloudera, HortonWorks, MarkLogic, Couchbase, etc. but this is not a definiFve list

•  All the big vendors (IBM, Oracle, EMC, SAP, etc.) also have big data offerings

•  Many of these technologies being deployed besides tradiFonal database technologies •  Very low cost as the soMware is free and by definiFon they use elasFc compute

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

18

Brussels 21 – 22

March ‘13 BI Tools In the Cloud •  Technical deployment of ReporFng and Data VisualisaFon Tools in the cloud is the easiest part of the process

•  Each product has a defined deployment model just as they do in an in-‐house data centre •  Obtain the servers (co-‐located, hosted or elasFc) and install the soMware and connect to the data servers

•  Some hosFng and infrastructure companies will provide pre-‐deployed soMware packages

•  The quesFon is does your soMware vendor understand offer an effecFve SaaS licencing model – most maintain their exisFng ‘per seat’ licence model rather than pay by use model

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

19

Brussels 21 – 22

March ‘13 Full Stack Cloud Providers •  Some companies have the depth of tools (ETL, Database, ReporFng) and faciliFes to provide a full BI in the cloud stack

•  IBM •  Datastage, Netezza, Cognos and various Big Data Technologies

•  MicrosoM •  SSIS, SQL Server 2012, ReporFng Services and Hadoop

•  Oracle •  Oracle Technology on Demand stack (Exadata, Discoverer, etc.)

•  SAP BI On Demand •  Business Objects reporFng and ETL with SAP HANA

•  In pracFce some are more mature than others in this area

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

21

Brussels 21 – 22

March ‘13 The Positive Impact of Cloud •  The provisioning of technology should be faster •  The cost base should be more predictable for compute resources

•  The cloud is an excellent way to trial new technologies and to run rapid developments in a Skunk Works approach

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

22

Brussels 21 – 22

March ‘13 The Negative Impact of Cloud •  Most BI operaFons have to deal with an IT department already – this doesn’t change and may even be more complex

•  Always consider the cost of moving large amounts of data from an in-‐house system to the cloud – this is oMen the breaking point for cloud based.

•  The reverse may also be true – your operaFonal systems may be put in the cloud before you have BI there, in which case you will be paying to download the data

•  There are many more operaFonal issues than technical issues around moving service provision to the cloud

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

23

Brussels 21 – 22

March ‘13 What doesn’t change •  Cloud and SaaS are just the latest iteraFon of cost management concepts (Timeshare, Outsourcing, etc.) for IT

•  They can help with the baseline costs of a BI project by providing and managing the commodity components of the environment

•  But …

•  Most BI projects are about an interacFon between the business and IT systems

•  They are about rapidly delivering informaFon in a constantly changing environment

•  <10% of the TCO over 10 years is in infrastructure – the rest is in people, development of ETL, support, change

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

24

Brussels 21 – 22

March ‘13 Geek & Poke’s View …

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

25

hfp://geekandpoke.typepad.com

Brussels 21 – 22

March ‘13 About Us Data Management & Warehousing is a UK based consultancy

that has been delivering successful business intelligence and data warehousing soluFons since 1995.

Our consultants have worked with major corporaFons around the world including the US, Europe, Africa and the Middle East.

We have worked in many industry sectors such as telcos, manufacturing, retail, financial and transport. We provide

governance and project management as well as experFse in the leading technologies.

13 Novem

ber

2012

© 2012 Da

ta M

anagem

ent &

Wareh

ousin

g

26

Brussels 21 – 22

March ‘13 Contact Us •  Data Management & Warehousing •  Website: hfp://datamgmt.com •  Telephone: UK +44 118 321 5930 •  US +1 716 406 3311

•  David Walker •  E-‐Mail: [email protected] •  Telephone: +44 (0) 7990 594 372 •  Skype: datamgmt •  White Papers: hfp://datamgmt.com/white-‐papers •  PresentaFons: hfp://datamgmt.com/presentaFons •  Blog: hfp://datamgmt.com/blog

13 Novem

ber

2012

© 2012 Da

ta M

anagem

ent &

Wareh

ousin

g

27

Brussels 21 – 22

March ‘13

BI Software as a Service & Cloud Strategies for Telcos Thank You

datamgm

t.com

© 2013 Da

ta M

anagem

ent &

Wareh

ousin

g

28

bi saas & cloud strategies for telcos

Documents

etl server

etl server

etl servers

term cost

deliver elt

low cost

community

warehousing