chris bradley evolve or die modelling not just for dbmss ... · pdf filestep 1: reverse...

Data Modelling:Understanding Business Information Needs

11th June 2009Christopher Bradley

www.ipl.com

Intelligent Business

About the speaker

29 years Data Management experience

MOD, Volvo, Thorn EMI, Coopers & Lybrand, IPL

Customers: BP, Audit Commission, MoD, Barclays

Experience: Data Modelling, Master Data Management, Data Governance, Information Architectures

Author & conference speaker

CDMP(Master), CBIP


Contents


History: Data Management growth :

Database developmentDatabase operation

1950-1970

Data requirements analysisData modelling

1970-1990

Enterprise data management coordinationEnterprise data integrationEnterprise data stewardshipEnterprise data use

1990-2000

Data qualitySecurity & ComplianceSOAAligning with the Business

2000-beyond

vs. the “new” view of Information• Web 2.0• Blogs• Mashups• Anyone can create data!


But it’s only useful for “new” Data Base Systems?

Data ArchitectureProduct CollaborationCommon Metadata

Integrating IT Roles through

Common Metadata

Models are the lingua franca


Background: Data Modelling’s promise ….

• "a single consistent definition of data"

• "master data records of reference"

• “reduced development time”

• “improved data quality”

• “impact analysis”

•…….

So why is it that in many organisations the benefits of data modelling still need to be “sold” and in others the

big benefits simply fail to be delivered?

No brainers?


Step 1: Document the business requirement and agree on high-level scope.

Step 1: Document the business requirement and agree on high-level scope.

BRD

Step 2: Create detailed business requirement document with subscriber data requirement, business process and business rules.

Step 2: Create detailed business requirement document with subscriber data requirement, business process and business rules.

BRD

Logical data model

Step 3: Understand and document the business keys, attributes and definitions from business subject matter experts. Create logical data model.

Step 3: Understand and document the business keys, attributes and definitions from business subject matter experts. Create logical data model.

Tables & foreign key constraints

Step 4: Verify the logical data model with the stakeholders. Apply technical design rules & create physical data model.

Step 4: Verify the logical data model with the stakeholders. Apply technical design rules & create physical data model.

Step 5: Implement using the created physical model. Step 5: Implement using the created physical model.

Approach to develop data models: Top-down (To-be)


Step 1: Reverse engineer the database schema that is already implemented.

Step 1: Reverse engineer the database schema that is already implemented.

Step 3: Find out foreign key relationships between tables, from IT subject matter experts & verify findings

Step 3: Find out foreign key relationships between tables, from IT subject matter experts & verify findings Tables & foreign key

constraints

Step 4: Document the meanings of columns, and tables from IT subject matter experts

Step 4: Document the meanings of columns, and tables from IT subject matter experts

Step 5: Try to understand the business meanings of probable attributes and entities that may be candidates for logical data model

Step 5: Try to understand the business meanings of probable attributes and entities that may be candidates for logical data model

Step 2: Profile data by browsing and analyzing the data from tables. Scan through the ETLs to find out hidden relationships & constraints.

Step 2: Profile data by browsing and analyzing the data from tables. Scan through the ETLs to find out hidden relationships & constraints.

A near logical data model

(accuracy unknown)

Approach to develop data models: Bottom-up (As-is)


So, it’s only useful for “new” Data Base Systems?


What needs to change

•Be relevant for the “new” technologies

•Make information available in a format users understand. • Re-purpose for the audience: Don’t show a business user a data

model!

•Make it real-time.• Information should updated instantaneously.

•Allow users to give feedback. • You’ll achieve common definitions quicker that way.

•Demonstrate benefits • Data modeling isn’t a belief system!

•But!...Remember the principles of data management


• Definition of data & consequently calls to / results from services is vital.

• Straight through processing can exacerbate the issue

• How traditional data principles still apply• Standard Definitions:

• What does the data mean? which definition of X (e.g. “cost of goods”)?

• Need to utilise the logical model and ERP models definitions • Structured Models

• Logical model drives XML formats

SOABus

BrokerAdaptersQueues

Messages areat the heart


Fundamentals: SOA

Message Based Interaction• All interaction between components is

through messages• Generally XML messages

Book

Book ISBN code

Amazon URL

Book name

Category

Publication date

Publisher

Recommended price

Book Authorship

Message: Book detailsMessage: Book details


Data integration & lineage

•Mashups & portals= easy assemblyof data …….… but is it correct?

•Still need the basics:• SOX lineage

requirements

• Repository based Data migration design = Consistency

• Legacy data take on

• Source to target mapping

• Reverse engineer & generate ETL

• Impact analysis


Data Lineage

•SOX lineage requirements

•Repository based Data migration design- Consistency

•Legacy data take on

•Source to target mapping

•Reverse engineer & generate ETL

• Impact analysis


Where do I need Data Lineage?

• Design of ETL processes

• Creation of Dimensional Models

• Transforming data to XML

• Workflow Design

• ….


ERP & packaged systems

• “We don’t need a data model – the package has it all”

• But, does it …• Meet your business requirements?

• Data Model will aid configuration / fit for purpose evaluation

• Have identical data structures & meanings as your legacy systems?• Data model will aid Data Integration, Legacy

Data take on and Master Data integration.

Intelligent BusinessData Governance 2.0


Intelligent BusinessData Governance 2.0



XML messages

•XML becoming all pervasive ….• Integration layers• “Standard” XML schemas• Off the shelf web services (WSDL)

•But, who is analysing the metadata & meaning?• Can utilise a logical data model for XML messages• But, XML message model = Hierarchic view of data model• Sub-models break up model into XML message chunk• Generate & customise XSD from sub-model• Import WSDL to capture metadata about data in the service

•Benefit = Accuracy, impact analysis, consistency

------------

------------

------------


MI / BI / DW

• Model Data requirements in Dimensional Model• Must start from understanding of existing data landscape

Product Category

Product

Product Sub Category

• Capture rich metadata via reverse engineering BW Info Cubes, BO Universes, …….

• Generate Star, Snowflake, Star flake schemas


Case study

Exploration & Production

Gas

RefiningAlternative Energy

Chemicals

LubricantsFuels Marketing & Retail


Avoid the Death star!


SAP model: 70,000 Islands of Information


SAP data in models

XXX

Profit Center

Profit Center Code

Controlling Area (FK)

Currency Code (FK)

NameProfit Center DescriptionStatus Code

Profit Center Group CodeValidity From DateValidity To DateAnalysis FromAnalysis ToLock Indicator

Joint VentureRecovery IndicatorBusiness CodeGRN CodeFunction Region CodeBTI CodeBusiness Unit CodeSegmentation CodeService Activity CodeTeam CodeSite CodeSegment Code

Currency

Currency Code

Currency Short DescriptionCurrency Long DescriptionISO CodeCurrency NameCurrency Type Key

Controlling Area

Controlling Area

Controlling Area NameChart of AccountsFiscal Year VariantCost Center Std HierarchyDocument Type

Cost Center CategoryCost Center Category Code

Description

Cost CenterCost Center Number

Cost Center Category CodCurrency Code (FK)Profit Center (FK)Cost Center NameCost Center DescriptionTeam CodeFunction Region CodeSite CodeBusiness Unit CodeService Activity Code

Intelligent Business25

Ston

gly

Agre

e

Dis

agre

e

79% 77%70%

55% 60%

4%0%

10%20%30%40%50%60%70%80%

Demonstrate benefits

We are not obtaining any benefits

We are obtaining benefit through use of a common modelling common modelling tooltool

We are obtaining benefit through utilisation of a common repositorycommon repository

We are obtaining benefit through use of common standards, common standards, guidelines & guidelines & processesprocesses

We are obtaining benefit through rere--use use of models & of models & artefactsartefacts We are obtaining benefit

through provision of central support & helpcentral support & help


Wiki feedback on models


Make models available in …..

The greatest change to

avoid extinction



Role of the Data Architect•How to gain Traction, Budget and Executive buy-in

• Be Visible about the program:• Identify key decision-makers in your organization and update them on your project and its value to the organization

• Focus on the most important data that is crucial to the business first! Publish that and get buy in before moving on. (e.g. start small with a core set of data)

•Monitor the progress of your project and show its value:• Define deliverables, goals and key performance indicators (KPIs) • Start small—focus on core data that is highly visible in the organization. Don’t try to “boil the ocean” initially.• Track and Promote progress that is made• Measure Metrics where possible

“Hard data” is easy (# data elements, #end users, money saved, etc.)“Softer data” is important as well (data quality, improved decision-making, etc.) Anecdotal examples help with business/executive users

“Did you realize we were using the wrong calculation for Total Revenue?”(based on data definitions)


3NF

Industry Culture

•DBAs, Data Architects and Executives are different creatures

DBA• Cautious• Analytical• Structured• Doesn’t like to

talk• “Just let me

code!”

Data Architect• Analytical• Structured• Passionate• “Big Picture” focused• Likes to Talk• “Let me tell you about

my data model!”

Business Executive• Results-Oriented• “Big Picture” focused• Little Time• “How is this going to help

me?”• “I don’t care about your

data model.”• “I don’t have time.”


Communicate Effectively

• Provide Information to uses in their “Language”• Repurpose information into various tools: BI, ETL, DDL, etc.• Publish to the Web• Exploit collaboration tools / SharePoint / Wiki …….• Business users like Excel, Word, Web tools

• Document Metadata• Data in Context (by Organization, Project, etc.)• Data with Definitions

• Provide the Right Amount of Information• Don’t overwhelm with too much information. For business users, terms and

definitions, might be enough.• Cater to your audience. Don’t show DDL to a business user or Business definitions

to a DBA.

• Market, Market, Market!• Provide Visibility to your project.• Talk to teams in the organization that are looking for assistance• Provide short-term results with a subset of information, then move on.


Ride those data cowboys out of town!

•Interpersonal skills

•Avoid methodology wars / bigots

•Nobody owes us a living!

•Data professionals constantly need to fight for their existence

•Professional certification


What needs to stay the same

•Modelling rigour• Logical and Physical Data models

• 3rd normal form

• Consistent definitions

•Standards & Governance

•Ownership

•Object reuse via a common repository


Contact details

Chris BradleyHead Of Information Management [email protected]+44 1225 475000


chris bradley evolve or die modelling not just for dbmss ... · pdf filestep 1: reverse...

Documents