selecting data management tools - a practical approach
TRANSCRIPT
P / 1
Considerations For Selecting Data Modelling Tools
C H R I S T O P H E R B R A D L E YI N F O R M A T I O N S T R A T E G Y A D V I S O R
P / 2
Considerations For SelectingData Modelling Tools
P / 3
Introduction:
My blog: Information Management, Life & Petrolhttp://infomanagementlifeandpetrol.blogspot.com
@InfoRacer
uk.linkedin.com/in/christophermichaelbradley/
Christopher Bradley
Information Strategy Advisor
+44 7973 184475
P / 5
P / 6
Introduction To Chris Bradley
Chris is a well known Information Strategy Advisor with 34 years
experience in the Information Management field, Chris works with
leading organisations including Celgene, Bristish Gas, HSBC, Total,
Barclays, ANZ, GSK, Shell, BP, Statoil, Riyad Bank & Aramco in Data
Governance, Information Management Strategy, Data Quality &
Master Data Management, Metadata Management and Business
Intelligence.
He is a Director of DAMA- I, holds the CDMP Master certification,
examiner for CDMP, a Fellow of the Chartered Institute of
Management Consulting (now IC) member of the MPO, and SME
Director of the DM Board.
A recognised thought-leader in Information Management Chris is
creator of sections of DMBoK 2.0, a columnist, a frequent
contributor to industry publications and member of standards
authorities.
He leads an experts channel on the influential BeyeNETWORK, is a
regular speaker at major international conferences, and is the co-
author of “Data Modelling For The Business – A Handbook for
aligning the business with IT using high-level data models”. He also
blogs frequently on Information Management (and motorsport).
P / 7
Recent PresentationsDAMA UK Webinar: February 2015; “An Introduction to the Information Disciplines of the
DAMA DMBoK”
Petroleum Information Management Summit 2015: February 2015, Berlin DE,
“How to succeed with MDM and Data Governance”
Enterprise Data & Business Intelligence 2014: (IRM), November 2014, London, UK “Data
Modelling 101 Workshop”
Enterprise Data World: (DataVersity), May 2014, Austin, Texas, “MDM Architectures & How to
identify the right Subject Area & tooling for your MDM strategy”
E&P Information Management Dubai: (DMBoard),17-19 March 2014, Dubai, UAE “Master
Data Management Fundamentals, Architectures & Identify the starting Data Subject Areas”
DAMA Australia: (DAMA-A),18-21 November 2013, Melbourne, Australia “DAMA DMBoK
2.0”, “Information Management Fundamentals” 1 day workshop”
Data Management & Information Quality Europe:
(IRM Conferences), 4-6 November 2013, London, UK
“Data Modelling Fundamentals” ½ day workshop:
“Myths, Fairy Tales & The Single View” Seminar
“Imaginative Innovation - A Look to the Future” DAMA Panel Discussion
IPL / Embarcadero series: June 2013, London, UK, “Implementing Effective Data
Governance”
Riyadh Information Exchange: May 2013, Riyadh, Saudi Arabia,
“Big Data – What’s the big fuss?”
Enterprise Data World: (Wilshire Conferences), May 2013, San Diego, USA, “Data and
Process Blueprinting – A practical approach for rapidly optimising Information Assets”
Data Governance & MDM Europe: (IRM Conferences), April 2013, London, “Selecting the Optimum Business approach for MDM success…. Case study with Statoil”
E&P Information Management: (SMI Conference), February 2013, London, “Case Study, Using Data Virtualisation for Real Time BI & Analytics”
E&P Data Governance: (DMBoard / DG Events), January 2013, Marrakech, Morocco, “Establishing a successful Data Governance program”
Big Data 2: (Whitehall), December 2012, London, “The Pillars of successful knowledge management”
Financial Information Management Association (FIMA): (WBR), November 2012, London; “Data Strategy as a Business Enabler”
Data Modeling Zone: (Technics), November 2012, Baltimore USA “Data Modelling for the business”
Data Management & Information Quality Europe: (IRM), November 2012, London; “All you need to know to prepare for DAMA CDMP professional certification”
ECIM Exploration & Production: September 2012, Haugesund, Norway: “Enhancing communication through the use of industry standard models; case study in E&P using WITSML”
Preparing the Business for MDM success: Threadneedles Executive breakfast briefing series,
July 2012, London
Big Data – What’s the big fuss?: (Whitehall), Big Data & Analytics, June 2012, London,
Enterprise Data World International: (DAMA / Wilshire), May 2012, Atlanta GA,
“A Model Driven Data Governance Framework For MDM - Statoil Case Study”“When Two Worlds Collide – Data and Process Architecture Synergies” (rated best workshop in conference); “Petrochemical Information Management utilising PPDM in an Enterprise Information Architecture”
Data Governance & MDM Europe: (DAMA / IRM), April 2012, London, “A Model Driven Data Governance Framework For MDM - Statoil Case Study”
AAPG Exploration & Production Data Management: April 2012, Dead Sea Jordan; “A Process For Introducing Data Governance into Large Enterprises”
PWC & Iron Mountain Corporate Information Management: March 2012, Madrid; “Information Management & Regulatory Compliance”
DAMA Scandinavia: March 2012, Stockholm, “Reducing Complexity in Information Management” (rated best presentation in conference)
Ovum IT Governance & Planning: March 2012, London; “Data Governance – An Essential Part of IT Governance”
American Express Global Technology Conference: November 2011, UK, “All An Enterprise Architect Needs To Know About Information Management”
FIMA Europe (Financial Information Management):, November 2011, London; “Confronting The Complexities Of Financial Regulation With A Customer Centric Approach; Applying IPL’s Master Data Management And Data Governance Process In Clydesdale Bank “
Data Management & Information Quality Europe: (DAMA / IRM), November 2011, London, “Assessing & Improving Information Management Effectiveness – Cambridge University Press Case Study”; “Too Good To Be True? – The Truth About Open Source BI”
ECIM Exploration & Production: September 12th 14th 2011, Haugesund, Norway: “The Role Of
Data Virtualisation In Your EIM Strategy”
Enterprise Data World International: (DAMA / Wilshire), April 2011, Chicago IL; “How Do You Want Yours Served? – The Role Of Data Virtualisation And Open Source BI”
Data Governance & MDM Europe: (DAMA / IRM), March 2011, London, “Clinical Information Data Governance”
Data Management & Information Management Europe: (DAMA / IRM), November 2010, London, “How Do You Get A Business Person To Read A Data Model?
DAMA Scandinavia: October 26th-27th 2010, Stockholm, “Incorporating ERP Systems Into Your Overall Models & Information Architecture” (rated best presentation in conference)
BPM Europe: (IRM), September 27th – 29th 2010, London,
“Learning to Love BPMN 2.0”
IPL / Composite Information Management in Pharmaceuticals: September 15th 2010, London,“Clinical Information Management – Are We The Cobblers Children?”
ECIM Exploration & Production: September 13th 15th 2010, Haugesund, Norway: “Information Challenges and Solutions” (rated best presentation in conference)
Enterprise Architecture Europe: (IRM), June 16th – 18th 2010, London: ½ day workshop; “The Evolution of Enterprise Data Modelling”
P / 8
Recent Publications
Book: “Data Modelling For The Business – A Handbook for aligning the business with IT using high-level data models”; Technics Publishing;
ISBN 978-0-9771400-7-7; http://www.amazon.com/Data-Modeling-Business-Handbook-High-Level
White Paper: “Information is at the heart of ALL Architecture disciplines”,; March 2014
Article: The Bookbinder, the Librarian & a Data Governance story ; July 2013
Article: Data Governance is about Hearts and Minds, not Technology January 2013
White Paper: “The fundamentals of Information Management”, January 2013
White Paper: “Knowledge Management – From justification to delivery”, December 2012
Article: “Chief INFORMATION Officer? Not really” Article, November 2012
White Paper: “Running a successful Knowledge Management Practice” November 2012
White Paper: “Big Data Projects are not one man shows” June 2012
Article: “IPL & Statoil’s innovative approach to Master Data Management in Statoil”, Oil IT Journal, May 2012
White Paper: “Data Modelling is NOT just for DBMS’s” April 2012
Article: “Data Governance in the Financial Services Sector” FSTech Magazine, April 2012
Article: “Data Governance, an essential component of IT Governance" March 2012
Article: “Leveraging a Model Driven approach to Master Data Management in Statoil”, Oil IT Journal, February 2012
Article: “How Data Virtualization Helps Data Integration Strategies” BeyeNETWORK (December 2011)
Article: “Approaches & Selection Criteria For organizations approaching data integration programmes” TechTarget (November 2011)
Article: Big Data – Same Problems? BeyeNETWORK and TechTarget. (July 2011)
Article “10 easy steps to evaluate Data Modelling tools” Information Management, (March 2010)
Article “How Do You Want Your Data Served?” Conspectus Magazine (February 2010)
Article “How do you want yours served (data that is)” (BeyeNETWORK January 2010)
Article “Seven deadly sins of data modelling” (BeyeNETWORK October 2009)
Article “Data Modelling is NOT just for DBMS’s” Part 1 BeyeNETWORK July 2009 and Part 2 BeyeNETWORK August 2009
Web Channel: BeyeNETWORK “Chris Bradley Expert Channel” Information Asset Management
http://www.b-eye-network.co.uk/channels/1554/
Article: “Preventing a Data Disaster” February 2009, Database Marketing Magazine
P / 9
Contents
_Context
_Approaches
_Evaluation Method
_Vendors and Products
_Summary
P / 10
1. Context
page 11
Is there more to life
than this?
is ais ais ais ais ais ais ais ais a
owns
is assigned
is owner of is owned by
belongs to has
registered at
classifies
has
classifies
transacted in
parent ofassigns assigns
performs
determines
payee-correspondent relationship
is granted
has
has
requires
Business Party
Business Person
Business Partner
Legal Entity
Address Type
Address
Bank Account
Contact
Counterparty Transportation Provider Financial Institution Exchange Inspector Broker Clearing HouseRegulatory Agency Rating Agency
Currency
Industry Classification
Industry Classification Scheme
Legal Entity Identification
Legal Entity Identifying Scheme
Bank Account Type
Internal Operating Unit
LE Ownership
Indirect Tax Registration
Tax Exemption License
Contact Usage
Contact Role
Payment Security Type
page 12
DATA
ARCHITECTUREMANAGEMENT
DATA
DEVELOPMENT
DATABASE
OPERATIONSMANAGEMENT
DATA SECURITY
MANAGEMENT
REFERENCE &
MASTER DATAMANAGEMENT
DATA QUALITY
MANAGEMENT
META DATA
MANAGEMENT
DOCUMENT & CONTENT
MANAGEMENT
DATA
WAREHOUSE & BUSINESS
INTELLIGENCE MANAGEMENT
DATA
GOVERNANCE
› Enterprise Data Modelling
› Value Chain Analysis
› Related Data Architecture
› External Codes
› Internal Codes
› Customer Data
› Product Data
› Dimension Management
› Acquisition
› Recovery
› Tuning
› Retention
› Purging
› Standards
› Classifications
› Administration
› Authentication
› Auditing
› Analysis
› Data modelling
› Database Design
› Implementation
› Strategy
› Organisation & Roles
› Policies & Standards
› Issues
› Valuation
› Architecture
› Implementation
› Training & Support
› Monitoring & Tuning
› Acquisition & Storage
› Backup & Recovery
› Content Management
› Retrieval
› Retention
› Architecture
› Integration
› Control
› Delivery
› Specification
› Analysis
› Measurement
› Improvement
Where does
Data Modeling
fit?
P / 13
Environmental Elements
Used with kind permission of DAMA-I
ACTIVITIES
• Phases, Tasks, Steps
• Dependencies
• Sequence and Flow
• Use Case Scenarios
• Trigger Events
ORGANIZATION & CULTURE
• Critical Success Factors
• Reporting Structures
• Management Metrics
• Values, Beliefs, Expectations
• Attitudes, Styles, Preferences
• Rituals, Symbols, Heritage
DELIVERABLES
• Inputs and Outputs
• Information
• Documents
• Databases
• Other Resources
TECHNOLOGY
• Tool Categories
• Standards and Protocols
• Section Criteria
• Learning Curves
PRACTICES
& TECHNIQUES
• Recognized Best
Practices
• Common Approaches
• Alternative Techniques ROLES &
RESPONSIBILITIES
• Individual Roles
• Organizational Roles
• Business and IT Roles
• Qualifications and Skills
GOALS &
PRINCIPLES
• Vision and Mission
• Business Benefits
• Strategic Goals
• Specific Objectives
• Guiding Principles
P / 14
Data Management Toolsprovide support for the data
management functions, including databases, modelling,
governance, quality and virtualisation.
Data Management Repositoriesare a class of Data Management
Tool used to store, version and disseminate the metadata
required to support the data management functions.
What are Data Management Tools and Repositories?
P / 15
What is Metadata?
P / 16
Logical Data Structures
Business Data Items
Concepts
Database Schemas
Modelling Patterns
Reports
Screens Data LineageNon-DBMS files
XML Schemas
What types of
Information
Metadata
can you
name?
Information Metadata
Lines represent shared
metadata elements,
and/or links between
metadata elements.
P / 17
The metadata archipelago, simplified
Information
Application
Architecture
Technical
Architecture
Organisation
Location People AssetsLocation People AssetsIT Service
Management
Project and Resource
Management
Business
Rules
Business
Processes
Web
Services
Development
Methods
SOA
IT Assets
Software HardwareSoftware Hardware
P / 18
None of this is new
In the early1980s, I used
the ICL Data Dictionary System (DDS) to model
and generate complete
applications.
Data Items were central
to this – they were used in
E-R models, screen
dialogues, IDMSX tables etc.
All this metadata was
linked and reusable.
Business Model
Computer Model
DataProcess
Adapted from a diagram on page 27 of the ICL Technical Journal, Volume 8 Issue 1, May 1992
P / 19
“Data /
Information” related Metadata
e.g.
“Process”
related Metadata
e.g.
ENTITY
EVENTPROCESS
TRANSFORMATION
FILE
SYSTEM
TABLE
DTD
FIELD
INDEX
SCHEMA
XSD PACKAGE
MODULE
TRIGGER
ROLE
SERVERCOLUMN
STORED PROCEDURE
Types Of Metadata
ATTRIBUTE
FUNCTION
WORKFLOW
PROJECT
PROGRAM
BUSINESS
RULE
RELATION-
SHIP
“Real
Business” World Metadata
“IT /
Systems” World Metadata
P / 20
What is a data modelling tool?
_Is it a metadata management tool, or a modelling tool?
_Modelling tools usually have pre-defined scope in terms of:
›The types of metadata supported
›The types of links it can create between metadata, within and between
models
_They may support:
›Model-driven architecture
›Model-driven analysis and design
P / 21
Possible metadata scope for data modelling
_Business Requirements
_Business Rules
_Business Processes
_Enterprise Architecture
_Glossary of Terms
_Object-Oriented Analysis and Design
_Conceptual And Logical Data Structures
_Physical Data Structures
›Databases
› XML Schemas
_Data movements, transformations and replications
P / 22
Tool Genres_Modelling
› Data Modelling Tools
› Model Management Tools
› Object Modelling Tools
› Process Modelling Tools
› XML Development Tools
_Development and
Maintenance
› System Management Tools
› Data Development and
Administration Tools
› Performance Management Tools
› Data Security Tools
› Collaboration Tools
› Application Frameworks
› Change Control Systems
_Database Systems
› Database Management Systems
› Data Integration Tools
› Database Administration Tools
› Identity Management
Technologies
_Repositories
› Meta-data Repositories
› Document Management Tools
› Configuration Management Tools
› Reference and Master Data
Management Tools
› Issue Management Tools
› Event Management Tools
› Image and Workflow
Management Tools
› Records Management Tools
_Analytical and Reporting
› Business Intelligence Tools
› Statistical Analysis Tools
› Analytic Applications
› Data Profiling Tools
› Data Quality Tools
› Data Cleansing Tools
› Report Generating Tools
› Data Governance KPI Dashboard
P / 23
We All Use Models
P / 24
We All Use Models
P / 25
Data Model Levels
E N T E R P R I S E
C O N C E P T U A L
L O G I C A L
P H Y S I C A L
S Y S T E M
IM
PL
EM
EN
TA
TI
ON
F
OC
US
CO
MM
UN
IC
AT
IO
N
FO
CU
S
P / 26
Why Produce A Data Model?
T O P R E A S O N S *
1. Capturing Business Requirements
2. Promotes Reuse, Consistency, Quality
3. Bridge Between Business and Technology Personnel
4. Assessing Fit of Package Solutions
5. Identify and Manage Redundant Data
6. Sets Context for Project within the Enterprise
7. Interaction Analysis: Complements
Process Model
8. Pictures Communicate Better than Words
9. Avoid Late Discovery of Missed Requirements
10. Critical in Managing Integration Between Systems
11. Pre-cursor to DBMS design / generate DDL
* DAMA-I Survey
P / 27
Enterprise
Conceptual
Logical
Physical
Enterprise vs. Conceptual vs. Logical
Agree basic
concepts and rules
Detail, may lead
to physical design
Big picture
Optimised for specific
technical environment
DIF
FER
EN
T P
ER
SP
EC
TIV
ES A
ND
LEV
ELS
OF D
ETA
IL F
OR
DIF
FER
EN
T U
SES
› Common understanding before progressing too far into detail
› Used to communicate with the Business› Overview: main entities, super types, attributes, and relationships
› Lots of Many to Many & multi meaning relationships › Relationships frequently show multiplicity of meaning › May be denormalised
› Non-atomic & multi-valued attributes allowed; no keys› Should fit on one page
› 20% of the modelling effort
› Detailed: ~ 5x Entities vs Conceptual model
› Detailed: Frequently pre-cursor to 1st cut physical (database) design› Detailed: Key input to requirements specification
› M:M relationships resolved: Intersection entities mostly have meaning› Relationship optionality added› Primary, foreign, alternate keys included
› Reference entities included› Fully normalized – no multi-valued, redundant, non-atomic attributes
› May be partitioned (sub-models)› 80% of the modelling effortC
ON
CEP
TUA
L /
LOG
ICA
L K
EY
DIF
FER
EN
CES
P / 28
Different Data Models For Different Audiences: Business
P / 29
Different Data Models For Different Audiences: Technical
P / 30
Why Data Modelling Is Critical
BUSINESSARCHITECTURE
Business Objectives & Goals
Motivations & Metrics
Functions, Roles, Departments
INFORMATIONARCHITECTURE
Enterprise Data Model
Conceptual Data Models
Logical Data Models
Physical Data Models
PROCESSARCHITECTURE
Overall Value Chain
High-Level Business Processes
Workflow Models
APPLICATION / SYSTEMSARCHITECTURE
Systems within Scope
High-Level Mapping
Business Services
Presentation Services (use cases)
P / 31
Why Data Modelling Is Critical
BUSINESSARCHITECTURE
Business Objectives & Goals
Motivations & Metrics
Functions, Roles, Departments
INFORMATIONARCHITECTURE
Enterprise Data Model
Conceptual Data Models
Logical Data Models
Physical Data Models
PROCESSARCHITECTURE
Overall Value Chain
High-Level Business Processes
Workflow Models
APPLICATION / SYSTEMSARCHITECTURE
Systems within Scope
High-Level Mapping
Business Services
Presentation Services (use cases)
The company is undertaking
a radical approach to enhance Customer
experience, service and satisfaction by providing
seamless multi-channel Customer access to all core
services
B U S I N E S S O B J E C T I V E S
I N F O R M A T I O N S E R V I C E S
B U S I N E S S S E R V I C E S
PRESENTATION SERVICES
B U S I N E S S P R O C E S S
ALL of the Architecture disciplines use the language
(and rules) of the data model
P / 32
Master &
Reference
Data
Management
Data
Warehousing
Business
Intelligence
Transaction
Data
Management
Semi-
Structured
Data
Management
Content &
Document
Management
DataIntegration &
Interoperability
DataQuality
Management
InformationLifecycle
Management
DataGovernance
DataAsset
Planning
EIMGoals &
Principles
Big
Data
Analytics
Data Models & Taxonomy's
MetadataManagement
Geospatial
Data
Management
Enterprise Information Management Framework
Business-Driven Goals Drive a
Robust Enterprise Information
Management Practice
EIM goals and strategies are business-driven for the
entire enterprise, underpinned by guiding principles
supported by senior managementProactive planning for the information lifecycle
including the acquisition, manipulation, access,
use, archiving & disposal of information
The identification of appropriate data
integration approach for business challenges
e.g. ETL, P2P, EII, Data Virtualisation or EAI.
The full lifecycle control and
management of information from
acquisition to retention & destruction
Organise information to align
with business & technical goals
using Enterprise, Conceptual,
Logical & Physical models
Roles, responsibilities, structures and
procedures to ensure that data
assets are under active stewardship
Processes, procedures and
policies to ensure data is fit
for purpose and monitored
Metadata capture,
management, & manipulation
to place data in business &
technical context
Manage diverse data sources across the
organisation from transaction data
management, to data warehousing and
business intelligence, to Big Data analytics
P / 33
P / 34
The Internet Of Things?
P / 35
Use a drawing tool?
P / 37
Why not use a drawing tool?
Drawing tools communicate ideas visually, and may capture some
additional information about some of the objects represented
each diagram stands alone: if a process appears on five diagrams,
there is no connection between those five symbols; you cannot
ensure that those five diagrams are consistent in how they depict
that process, nor is there a way of knowing whether or not the
process also appears on a sixth diagram that you haven’t been told
about
P / 38
Why not use a drawing tool?
There is no link from that process to any objects that were
generated from it, and no link to any associated objects
Objects appear to live in a disconnected world. When it comes to
managing change, that disconnected world gets very
uncomfortable
_In conclusion:
Drawing tools do not manage metadata
P / 39
A typical Business Process ModelThis was created in a
modelling tool, but
could equally have
been drawn in a
‘drawing tool’.
P / 40
Can a drawing tool be customised?Stereotypes have been used in
the process model, to add new
types of objects, and
specialisations of standard
objects, such as Resources,
complete with custom symbols
P / 42
0,n
Each New Entity must have one and only one Authorship.
Each Authorship may have one or more New Entity.1,1
1,1
Each Book Role may have one or more New Entity.
Each New Entity must have one and only one Book Role.
0,n
0,1
Each Gender may classify one or more Person.
Each Person may claim at most one Gender.
0,n
0,1
Each Person must play one or more Book Role.
Each Book Role may be played by at most one Person.
1,n
1,1
Each Book Role must be recognised via one or more Authorship.
Each Authorship must be attributed to one and only one Book Role.
1,n
Authorship
Person Name
Book Title
Royalty Percentage
Characters (100)
Characters (100)
Number (2)
Primary ID <pi> Primary Identifier
Relationship 'Author-Authorship'
Relationship 'Relationship_6'
Book Role
Person Name
Book Role Type
Pseudonym
Characters (100)
Number (2)
Characters (100)
Primary ID <pi> Primary Identifier
Relationship 'Person Role'
Relationship 'Author-Authorship'
Relationship 'Relationship_5'
Person
Person Name
Gender Code
Characters (100)
Characters (60)
Primary ID
Alt
<pi>
<ai>
Primary Identifier
Relationship 'Reference_3'
Relationship 'Person Role'
Gender
Gender Code
Gender Name
Characters (60)
Characters (256)
Key_1 <pi> Primary Identifier
Relationship 'Reference_3'
New Entity
Aut_Person Name
Book Title
Person Name
Characters (100)
Characters (100)
Characters (100)
Identifier_1 <pi> Primary Identifier
Relationship 'Relationship_5'
Relationship 'Relationship_6'
Can a drawing tool mimic a different tool?Customised to look like it
was produced by a
different tool
P / 43
Can a drawing tool create matrices?
Like this editable matrix
showing links between LDM attributes and domains
Or this editable matrix
showing links between LDM
attributes and Business Rules
P / 44
Can a drawing tool link models?
This visual representation shows you the ‘generation links’ between a
Logical Data Model and a Physical Data Model
P / 45
Can a drawing tool trace dependencies?
Like the impact and lineage analysis for this column, which is:
• used by two indexes and one key
• linked to a Business Term and a Requirement
• governed by a domain shared from another model
• a foreign key
and was generated from a LDM attribute
P / 46
Can a drawing tool create an OLAP Cube?
Like converting the tables and
references at the top to the
design of an OLAP Cube at the
bottom.
Can it also generate the OLAP
Cube, complete with the
scripts used to transfer the data
from the relational database?
Can it reverse-engineer an
OLAP Cube?
FK_MONTH_RELATIONS_YEAR
FK_BOOK_SAL_RELATIONS_MONTHFK_BOOK_SAL_RELATIONS_PUBLICAT
Publication
Book Title
Publication Media Type Code
Publication Date
Dollar List Price
ISBN
Page Count
Primary Author Name
Primary Author Pseudonym
Primary Author Royalty Percent
age
CHAR(100)
NUMBER(2)
DATE
NUMBER(5,2)
NUMBER(13)
NUMBER(4)
CHAR(100)
CHAR(100)
NUMBER(2)
<pk>
<pk>
<i>
<i>
not null
not null
null
not null
not null
null
not null
null
null
Book Sales
Year Code
Month Code
Book Title
Publication Media Type Code
Gross Sales Value Amount
NUMBER(2)
NUMBER(2)
CHAR(100)
NUMBER(2)
NUMBER(5,2)
<pk,fk1>
<pk,fk1>
<pk,fk2>
<pk,fk2>
<i1,i2>
<i1,i2>
<i1,i3>
<i1,i3>
not null
not null
not null
not null
not null
Month
Year Code
Month Code
Month Description
CHAR(60)
CHAR(60)
CHAR(256)
<pk,fk>
<pk>
<i1,i2>
<i1>
not null
not null
null
Year
Year Code
Year Description
CHAR(60)
CHAR(256)
<pk> <i> not null
not null
Book Sales - Year_Month
Book Sales - Publication
MeasureBook Sales
Gross Sales Value Amount
Year Code
Month Code
Book Title
Publication Media Type Code
Year_Month
Year Year Code
Year Description
Month Year Code
Month Code
Month Description
<h:1>
<h:2>
<h:3>
Hierarchy_1 <Default> <h>
Publication
Book Title
Publication Media Type Code
Publication Date
Dollar List Price
ISBN
Page Count
Primary Author Name
Primary Author Pseudonym
Primary Author Royalty Percentage
<h:1>
<h:2>
Hierarchy_1 <Default> <h>
Attributes
Hierarchy
P / 47
Can a drawing tool manage versions, branching and configurations?
Repositories can support multiple
versions of models, branching to
support multiple environments, and
the grouping of related models to
form configurations.
Permissions can be varied by
repository folders, branches, projects,
and models.
P / 48
Can a drawing tool provide access via a Portal?
P / 49
Can a drawing tool merge or compare models?
P / 51
Can a drawing tool present a different UI to different users?
Can a drawing tool give you control of naming standards?Can a drawing tool
tell you which diagrams an object is in?
Does a drawing tool allow you to add functionality?
Can a drawing tool generate HTML or RTF reports?
P / 52
2. Approaches
_3 different approaches to
selecting tools
P / 53
Approaches
P / 54
TacticalEnterprise Best of Breed
Approaches
P / 55
Tactical Tool (E.g. Visio/PowerPoint)
_Advantages
›Best quick fit for the job
›Just-in-time, meets immediate need
›Quick (often zero) selection and procurement process
›Already have expertise (“Use what you know”)
_Disadvantages
›Does not meet strategic, long-term needs
›No metadata repository
›Lack of interoperability – difficulty in integrating data for holistic view
›Limited re-use / leverage possibilities
P / 56
Enterprise Toolset
_Benefits
›Multiple modelling capabilities (e.g. Data, Process, Enterprise, Technology …..)
›Artefacts linked & shared in Enterprise class repository
›Skills can be shared across the enterprise
›Modelling artefacts stored and used consistently
›Aligned with strategic goals
_Disadvantages
›One size does not fit all, one “modelling” area capability may be sub-optimal
›Easy to over-provision, unused functionality
›Protracted, expensive selection and procurement process
›May force unfamiliar techniques on users
the needs of the many outweigh the needs of the few
P / 57
Best of Breed Tooling
_Benefits
›Best fit for a single modelling discipline
›Skills can be shared across the enterprise
›Volume license discounts
›Modelling artefacts in one type stored and used consistently
_Disadvantages
›Exchange of artefacts between modelling tools
›Potentially requires a 3rd party repository
›Potentially requires support from multiple vendors for full modelling
reach
›Great care required in selection and procurement process
P / 58
Some enterprise capable, others suitable for small-scale modelling only
Some specific to data modelling,
others cover other aspects of
modelling, e.g. process modelling,
business architecture, enterprise
architecture
Some provide data modelling
capability only as part of other
functionality, e.g. part of database
programming IDE
Some targeted at particular database
platforms, whilst others are cross-
platform
A wide range of tools available
Data Modelling Tools
P / 59
Each tool has strengths and
weaknesses
Consider capabilities in non-
data modelling areas too
Differences may not be obvious
from vendor marketing
Take time to evaluate
capabilities to select tool that
meets your needs
Summary
Data Modelling Tools
P / 60
3. Evaluation method
P / 61
Outline Evaluation Method
P / 62
Steps 1-6
1. Establish TOR &Identify
requirements
2. Identifyconstraints
3. Tailorevaluation criteria
4. Assignweightings to
evaluation criteria
5. Compile an initiallist of candidate
products
6. Evaluateproducts
• Evaluation Terms of Reference• Statement of client requirements:
• High level functional requirements• Performance & technical requirements• Commercial requirements
• Key decision making criteria.
Key documents and deliverables :
• Statement of client constraints:E.g. hardware; existing software; operating system; cost; migration effort & timescale, risk, staff skill
• Tailored, product & vendor evaluation questionnaires
• Tailored, unweighted product & vendor evaluation spreadsheet(s)
• Weighted product & vendor evaluation spreadsheet(s)
• Short list of candidate products• Product elimination or exclusion reasons• Supplier pre-qualification questionnaires and/or
briefings prepared and issued• Progress log for tracking status of
communications with suppliers.
• Issued vendor questionnaires,• Vendor visit reports,• User visit reports
Step
From 9 (sensitivity analysis)
P / 63
Steps 7-10
To 4 (assign weightings)
7. Apply theevaluation criteria
against theproducts
8. Score and rankthe products
9. Performsensitivity and
"what if" analyses
10. Present thefindings
* Reduce product list* If necessary re-addressweightings* More detailed evaluationof reduced product list
• Updated product and vendor evaluation spreadsheet(s)
• In house demonstration / proof of concept report
• Scored & ranked spreadsheets• Recommendation and rationale for exclusion /
adoption of products
• Obtain further information• Confirm recommendation
• Product findings management summary• Evaluation spreadsheet(s)• Vendor product literature• Product pros & cons• Recommend product + implications for use• Recommended next steps (e.g. procurement,
negotiate terms, POC, installation, etc)
P / 64
Why follow an evaluation approach?
›Often several different tools are already in place for different user communities.
› Evaluation must be seen to be fair and equitable, and address the priority
requirements.
›A selection cannot be effective if a checklist is the sole justification of whether
a product or supplier is suitable or not.
›A product rating highly on say the technical evaluation criteria may not always
be the best one for your particular environment, i.e. staff skill levels, attitude to
risk etc.
› Selection must not be solely based on how “good” a product is, but rather on
how well suited it is to business needs.
› The chosen product may not necessarily be the “best” on the market but it will
be the one that best meets your requirements, and yields the optimum
benefits.
›Critical that requirements are fully understood, documented and prioritised.
› Take advice to highlight the implications of requirements (or sometimes lack of
them) before the evaluation progresses too far.
P / 65
Evaluation Criteria
_Sources of knowledge
_Weighting approach
_Rating & Scoring approach
P / 66
Evaluation criteria - sources
I N F O R M A T I O N N E C E S S A R Y T O S C O R E T H E D E T A I L E D Q U E S T I O N S F O R E A C H
P R O D U C T W I L L B E D E R I V E D F R O M M A N Y P L A C E S I N C L U D I N G : -
›Actual experience of the evaluation team;
› Experience of other client staff;
› Reference visits, talking to existing users;
› Liaison and discussion with the vendors;
› Trying out the vendor training programs;
› Feedback from the press and the general marketplace;
› Evaluation reports such as those from Gartner, Ovum and Forrester;
›Consultancy support /previous evaluation projects;
› Benchmarking / trial licence use;
›Gut feel;
›……..
P / 67
Evaluation criteria - weighting
A S S I G N W E I G H T S S U C H T H A T T H E S U M O F T H E W E I G H T S F O R A L L
O F T H E C R I T E R I A I N T H E S U B - S U B - S E C T I O N E Q U A L S 1 0 0 ;
› The same technique is applied at the sub-section level to indicate the relative
importance of the sub-section within a section;
› The same technique is then applied at the section level to indicate the relative
importance of that section for a particular product type.
› If the evaluation is not just concerned with a single product type but with a complete
package of products from a single vendor, an additional level of weighting will be
used to indicate the relative importance of each product type within the complete
evaluation package.
› Each of the team members, in isolation, should apply weightings to the criteria.
› Review the complete set of weightings to identify any major discrepancies, these
should be discussed and finally agreed to give an overall team weighting.
›Undertake the weighting approach before any detailed investigation is undertaken.
P / 68
Section Heading Sub section Question Questionweight
Sub section weight
Section weight
1. Vendor 40
1.1 Co details (general) Several questions / possibly sub-sub sections 10
1.2 Co details (product) Several questions / possibly sub-sub sections 10
1.3 Product plans Several questions / possibly sub-sub sections 15
1.4 Support Services / training
Several questions / possibly sub-sub sections 25
1.5 Commercials Several questions / possibly sub-sub sections 40
100
2. Data Modelling Functional Capabilities 20
2.1 Model levels & types supported
40
Enterprise & Conceptual model support 30
Logical model support 30
Physical model support 20
Dimensional model support 20
100
2.2 Methods & notations Several questions / possibly sub-sub sections 20
2.3 Sub model support Several questions / possibly sub-sub sections 20
2.4 Validation & standards
Several questions / possibly sub-sub sections 20
100
3. Interfaces & Integration Several questions & sub sections 10
4. Management , Collaboration & Extension Several questions & sub sections 10
5. Repository Several questions & sub sections 10
6. Non functional Several questions & sub sections 10
TOTAL 100
P / 69
Evaluation criteria - rating
9-10 The product adequately satisfies the criterion and, in addition, has significant
usable advantages. A nine or ten is assigned based on the strength of these
advantages.
8 The product adequately satisfies the criterion, but has no particular strong points
or weak points.
5-7 The product meets the criterion but has some weak points. In certain cases, the
weak points can be somewhat counterbalanced against identified strong points.
In other instances, the solution could provide functional capabilities but will be
penalized for the ease of use of the features.
1-4 The product meets the criterion on a marginal basis only, and, in addition, has
significant weak points. In some instances, a weakness in a particular area can
be viewed as a technical constraint in the use of the product. It will also imply
that user programming or other non trivial work rounds will be required to meet a
particular requirement.
0 The product does not meet the criterion. Total user implementation will be
required.
Each member of the team, in isolation, performs the rating of the products.
Review to identify any discrepancies.
Discuss amongst the evaluation team & averaged to give a team rating.
Rating for each product is multiplied by weighting yielding a score for that product.
Scores are computed for all criteria; summarized to the sub-section, section, product type
and total level.
P / 70
Evaluation Categories
_Example data modelling tool evaluation categories
P / 71
Evaluation criteria (vendor)
T E X T
_Company Details (General)
› Size, Turnover, Geographic reach ….
_Company Details (Product Division)
› Product family tree, history, acquisition ….
_Product Plans / Philosophy
› Product direction, enhancement approach, vision ….
_Support, Services, Training
› Problem solving, fixes, professional services, training ….
_Contractual Terms / Commercial
› Licensing types, concurrent, named users, discount….
P / 72
P / 73
P / 74
Master &
Reference
Data
Management
Data
Warehousing
Business
Intelligence
Transaction
Data
Management
Semi-
Structured
Data
Management
Content &
Document
Management
DataIntegration &
Interoperability
DataQuality
Management
InformationLifecycle
Management
DataGovernance
DataAsset
Planning
EIMGoals &
Principles
Big
Data
Analytics
Data Models & Taxonomy's
MetadataManagement
Geospatial
Data
Management
Reference Architecture W H A T I S T H E C O M P L E T E
P A C K A G E R E Q U I R E D T O B E
H I G H L Y E F F I C I E N T W . R . T .
D A T A M O D E L L I N G A C R O S S
T H E E N T E R P R I S E ?
EIM goals and strategies are business-driven for the
entire enterprise, underpinned by guiding principles
supported by senior managementProactive planning for the information lifecycle
including the acquisition, manipulation, access,
use, archiving & disposal of information
The identification of appropriate data
integration approach for business challenges
e.g. ETL, P2P, EII, Data Virtualisation or EAI.
The full lifecycle control and management of information from
acquisition to retention & destruction
Organise information to align
with business & technical goals
using Enterprise, Conceptual,
Logical & Physical models
Roles, responsibilities, structures and
procedures to ensure that data
assets are under active stewardship
Processes, procedures and
policies to ensure data is fit
for purpose and monitored
Metadata capture,
management, & manipulation
to place data in business &
technical context
Manage diverse data sources across the
organisation from transaction data
management, to data warehousing and
business intelligence, to Big Data analytics
P / 75
Data Modeling Tool Features
_With life and data modeling tools, you
get what you pay for.
_The more you pay, the more features
are provided by the product.
_The wider your use of data models
within the organization, the more
features you tend to need.
P / 76
Evaluation criteria (tools)
R E Q U I R E M E N T S / E V A L U A T I O N C A T E G O R I E S
_Data Modelling Functional Capabilities
_Interfaces & Integration
_Management & Collaboration
_Repository
_Non-functional
P / 77
1. Data Modelling Functional Capabilities
Model levels & type supported
Enterprise, Conceptual, Logical, Physical, Dimensional
Methods / Notations
ER, UML Class Diagrams, Barker, Object Relational
Sub models
Model partitioning, Model linking, Generalisation &
Inheritance
Validation, Standards
Model validation, Own rules
P / 78
“A Picture is Worth a Thousand Words” Examples of High-Level Data Models
P / 79
Is Notation Important?
_Many Notations can be used to express a high-level data model
_The choice of notation depends on purpose and audience
_For data-related initiatives, such as MDM and DW:
›ER modeling using IE (Information Engineering) is our choice of notation
› It is important that your high-level model uses a tool that can generate DDL, or can import/export with a tool that can
›A repository-based solution helps with reuse and standards for enterprise-wide initiatives
_We’re not producing art
P / 80
Blog: Information Management, Life & Petrol
http://infomanagementlifeandpetrol.blogspot.com
P / 81
Merise Notation
0,n
(1,1)0,n
1,1
0,n
Inheritance_1
Entity_1
Sub Type
Super Type
Relationship_1
Relationship_2
Entity_4
P / 82
Barker Notation
Relationship_2
Relationship_1Entity_1
Super Type
Sub Type
P / 83
IDEF1X Notation
Relationship_2
Relationship_1
Inheritance_1
Entity_1
Sub Type
Super Type
P / 84
Information Engineering
Relationship_2
Relationship_1
Inheritance_1
Entity_1
Sub Type
Super Type
P / 85
1. Data Modelling Functional Capabilities: Usability_C o n t r o l o v e r t h e w i n d o w s a n d
t o o l b a r s d i s p l a y e d , a n d t h e i r p o s i t i o n
a n d s i z e
_A u t o - l a y o u t o f s e l e c t e d p a r t s o f a
d i a g r a m o r a c o m p l e t e d i a g r a m
_C o n t r o l o f t h e p l a c e m e n t o f s y m b o l s
o n a d i a g r a m b y t h e a n a l y s t
_C o n t r o l o v e r t h e s t y l e a n d c o n t e n t o f
s y m b o l s b y t h e a n a l y s t
_F l e x i b l e e d i t i n g c a p a b i l i t i e s
_F l e x i b l e d i a g r a m p r i n t i n g c a p a b i l i t i e s
_R e p l a c e s e l e c t e d t e x t i n o b j e c t n a m e s
a n d o t h e r p r o p e r t i e s
_A n n o t a t e d i a g r a m s w i t h a d d i t i o n a l
s y m b o l s t o i m p r o v e c o m m u n i c a t i o n
_E x p e r t , t i m e l y s u p p o r t a v a i l a b l e
P / 86
P / 87
P / 88
2. Interfaces & Integration
Generation capabilities
DBMS, XML, Cubes, Rules, Stored Procedures,
Triggers, SOA,
Roll up / down
Reverse Engineering
ER <> Class, XML, DBMS
Integration
Business process, EA, Dev tools
MetaData Exchange
XMI, Direct tool interfaces,
Missing component reporting
P / 89
2. Interfaces and Integration
_ I m p o r t e x i s t i n g d a t a m o d e l s c r e a t e d b y
o t h e r t o o l s
_ R e v e r s e e n g i n e e r e x i s t i n g d a t a a r t e f a c t s ,
s u c h a s X M L s c h e m a s a n d d a t a b a s e
s c h e m a s , i n t o p h y s i c a l d a t a m o d e l s
_ G e n e r a t e o r u p d a t e a n e x t e r n a l d a t a
a r t i f a c t , s u c h a s a n X M L o r d a t a b a s e
s c h e m a , f r o m a d a t a m o d e l
_ C r e a t e o r u p d a t e a m o d e l b a s e d u p o n
i n f o r m a t i o n h e l d i n s p r e a d s h e e t s
_ C o m p a r e t w o d a t a m o d e l s , a n d u p d a t e o n e
o r b o t h m o d e l s a s a r e s u l t ( t h i s m a y a l s o b e
r e f e r r e d t o a s m e r g i n g m o d e l s )
_ C o m p a r e a d a t a m o d e l w i t h a n e x i s t i n g
d a t a a r t i f a c t , s u c h a s a n X M L o r d a t a b a s e
s c h e m a , a n d u p d a t e t h e m o d e l a n d / o r t h e
d a t a a r t i f a c t a s a r e s u l t
_ E x p o r t d a t a m o d e l s i n a f o r m a t t h a t c a n
b e o p e n e d b y o t h e r t o o l s
_ B u i l d c r o s s - r e f e r e n c e s o r t r a c e a b i l i t y
l i n k s b e t w e e n d a t a m o d e l o b j e c t s a n d
o b j e c t s d e f i n e d i n o t h e r t y p e s o f
m o d e l s , s u c h a s b u s i n e s s p r o c e s s o r
e n t e r p r i s e a r c h i t e c t u r e m o d e l s
_ I n t e g r a t i o n w i t h p o p u l a r d e v e l o p m e n t
e n v i r o n m e n t s
_ I n t e g r a t i o n w i t h p r o c e s s a n d p o r t f o l i o
m a n a g e m e n t w o r k f l o w s , a n d w i t h I T
c o n f i g u r a t i o n m a n a g e m e n t
_G e n e r a t e s c r i p t s t o m a n a g e t h e
m o v e m e n t o f d a t a t h r o u g h t h e a r c h i v i n g
c y c l e , o r d a t a m o v e m e n t s
P / 90
P / 91
3. Management, Collaboration & Extension
Collaboration
Multi team usage, Team working capabilities, Workflow
Ease of Use
Global standards, Search, Usability
Import, Merge, Compare
Types of import, Conflict resolution, Comparison types,
Generate delta vs. total DDL
Extensibility
User defined properties, New metadata types, Open API, Macros,
Published Object Model, Published Repository Model
Security & Control
Access Control, Content Control, Functionality Control,
Version control, Auditing
Reporting
Definition, Publishing, Web / Intranet Portal
P / 92
3. Management, Collaboration & Extension
_E x t e n d t h e t o o l ’ s u n d e r l y i n g
d a t a m o d e l , a l l o w i n g a n a l y s t s
t o c h a n g e t h e w a y i n w h i c h
m o d e l o b j e c t s a r e d e f i n e d a n d
t o d e f i n e n e w t y p e s o f m o d e l
o b j e c t s
_E x t r a c t i n f o r m a t i o n f r o m m o d e l
o b j e c t s f o r p u b l i c a t i o n i n
v a r i o u s f o r m a t s , s u c h a s H T M L
a n d d o c u m e n t s
_P r o v i d e a c c e s s t o t h e c o n t e n t
o f m o d e l s v i a a p r o g r a m m a b l e
i n t e r f a c e , t o p r o v i d e a
m e c h a n i s m f o r t h e a u t o m a t i o n
o f r e p e t i t i v e t a s k s , a n d t o
e x t e n d t h e f u n c t i o n a l i t y
p r o v i d e d b y t h e t o o l
_I n t e g r a t e w i t h L D A P / A c t i v e
D i r e c t o r y f o r u s e r
a u t h e n t i c a t i o n
P / 93
P / 94
& version control & auditing …
P / 95
4. Repository
Tool Integration & Data Definition
Model repository, Active vs Passive, Types, Validation
Architecture
Stand alone tool, Repository architecture, Reporting, CWM
Extensibility
Own metadata, non “data model” metadata
P / 96
P / 97
Example Repository Uses
Common requirements?
•Metadata of ROR’s
•Models of ROR
•Business definitions
•Ownership
•Stakeholders
•Provenance
•Business roles
•Security
•Version control
•Synonym support
•Reporting
• …….
SoA
Services
Directory
Data
Distribution
Services
(eg DV)
Enterprise
Data
Catalogue
Data
Modelling
Repository
P / 98
5. Non Functional
Cloud Data Modelling Service
Provision & use in Cloud, DMaaS, Cloud licensing
User Group
Vendor support, How active
Documentation
Product, Quick Start standards
P / 99
0
200
400
600
800
1000
1200
We
igh
ted
To
tal
Weighted Total Summary
Soft Issues
Data Modelling Respository
Management Features
Interfaces and Integration
Modelling
P / 100
Enterprise Repository
Data Modelling
Business Process
Technology
Infrastructure E
Enterprise Repository
Data Modelling
Business Process
Technology
Infrastructure A
Enterprise Repository
Data Modelling
Business Process
Technology
Infrastructure P
Enterprise Repository
Data Modelling
Business Process
Technology
Infrastructure S
P / 101
4. Vendors and Products
P / 102
Process Modelling Tools
P / 103
Data Integration Tools
P / 104
Object Modelling Tools
P / 105
Business Intelligence/Reporting Tools
P / 106
Data Quality/Profiling/Cleansing Tools
P / 107
Data Modelling Tools
http://www.information-management.com/media/pdfs/MySoftForge.pdf
http://en.wikipedia.org/wiki/Comparison_of_data_modeling_tools
Dezign
Modelright
Others
See databaseanswers.com
P / 108
Metadata Repositories
RochadeAdaptiveMetadata ManagerDatafluxetc
P / 109
P / 110
5. Summary
P / 112
Summary
3 approaches to evaluating tools
An evaluation method provides auditability
Weight before scoring
Get stakeholders involved
It’s YOUR requirements, NOT an academic study
Information is at the heart of all architecture disciplines
There's more to modelling than just data
P / 113
And finally
The quality and reliability of comparative evaluations issued by vendors varies
significantly
the worst we’ve seen (very recently) was ‘unofficial’, presumably produced by a sales
rep for a particular customer. It was completely unprofessional, the sole intention was to
rubbish a competitor, no matter how true it was. For example,
claiming that the other tool doesn't support feature Y, just because it doesn’t have a
feature called Y - in this case, it does support that feature, just happens to give it a
different name
missing information – “my tool supports both relational and dimensional modelling” –
doesn’t mention the fact that the other tool also supports both of them
apparent hearsay – throwaway comments such as “they say that my tool can
leverage colour better than tool Y” with no supporting information
it’s very difficult to produce an unbiased and detailed comparison of tools, as very few
people know the target tools in sufficient detail
take everything with a huge pinch of salt – take time to come to your own conclusions
H O W M U C H C A N Y O U R E L Y O N T O O L C O M P A R I S O N S F R O M V E N D O R S ?
P / 114
Are you going this year? It’s at October 5th-7th at Chapel Hill, North Carolina.
Find out more or register at http://datamodelingzone.com
Alternatively, go to Hamburg, Germany September 28th-29th.
To receive a discount of 20% when you register, use this discount code –
MCGEACHIE
P / 115
Contact:
My blog: Information Management, Life & Petrolhttp://infomanagementlifeandpetrol.blogspot.com
@InfoRacer
uk.linkedin.com/in/christophermichaelbradley/
Christopher Bradley
Information Strategy Advisor
+44 7973 184475