data modelling as a service in a super major

1

if you use this format

with a picture in

the vertical-stripe

format, then adjust

the RH edge of the title bar to be just L of the stripe.

Data Modelling as a Servicein a super major

Our journey

ER/Sig DAMA Boston, 7th March 2007Christopher Bradley

2


with a picture in

the vertical-stripe

format, then adjust


[email protected]

Contents

The challengeOur solution

1. Self service user administration & provisioning for internal users

2. Detailed reporting of ER/Studio and Repository usage for user tracking (and chargeback)

3. Automated Model publishing

Issues / next steps

3 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file \zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+

The ChallengeCompany is a diverse federated organisation – culture doesn’t lend itself towards big central control teamsCurrent position:Data modelling undertaken to different degrees in different Segments & FunctionsVariety of tools & techniques used to define DATA models

•ARIS, ERWin, System Architect, KMDM, Enterprise Architect, Power Designer, Rational, PowerPoint, Visio, …… others?

Projects encounter common cross business data concepts, but create their own models & definitionsMassive benefit in reuse of common models & definitions (including standard industry models)

•Plus benefits in interoperability, & efficiency through common approaches.


The Challenge

However …..Wanted to introduce data services including Data Modelling as a Service (DMaaS)

•Environment & service not just a tool. DMaaS includes: Training & Education, Project support & mentoring, Community Of Interest, Documentation & “best practice” guidelines, Technology support (inc ER/Studio, macros, SharePoint etc).

Don't know how many / where users are•very volatile user population; •100's of modellers, •1000's of analysts •1000’s business users wanting to read / view models

Standard common models (e.g. Master Data) need to be developed & then consumed by all projects.

•Therefore common shared repository & standard DME


[email protected]

1. Self service user admin & provisioning


Self service user administration & provisioning for internal users to:•register for ER/Studio•gain & change repository permissions

•repository password change•licence server access•view registered users / managers (& members) of teams can see who’s registered

1: Self Service


•View registered users / managers (& members) of teams can see who’s registered

Self Service


Lets managers know who has registered (or who has not) on their teamLets users verify they are registered correctlyLets users see other members of the internal data modelling community

Self Service – Example: View users


•Register new user

Self Service


Self Service – Example: Register New User

New user request submitted from SharePoint

Request received and validated against Active Directory

User created in database

User created in Repository

User given default permissions

Welcome email sent

User only has to provide business area and MAC address - form is quick and simple to complete

Repository

Repository Server

Data Modelling Environment SharePoint

Email Client

Active Directory

ER/Studio Application

Repository Access Web Service

UserDatabase

Firewall

Data Modelling EnvironmentSupport Application








Welcome email sent

User name, email address and contact information is retrieved from the BP Global Address List using

Active Directory

Repository

Repository Server


Email Client

Active Directory



UserDatabase

Firewall









Welcome email sent

The user is added to the local user database

Repository

Repository Server


Email Client

Active Directory



UserDatabase

Firewall









Welcome email sent

The user is created in the Repository using ActiveX automation and assigned a randomly generated

password

Repository

Repository Server


Email Client

Active Directory



UserDatabase

Firewall









Welcome email sent

Permissions cannot be assigned using ActiveX automation so a bespoke web service is used to

modify the database directly

Repository

Repository Server


Email Client

Active Directory



UserDatabase

Firewall




An email is created, based on a template heavily customised based on the users personal

information and licenses requested






Welcome email sent

Repository

Repository Server


Email Client

Active Directory



UserDatabase

Firewall



Self Service

Repository

Repository Server


Email Client

Active Directory



UserDatabase

Firewall



Request received and validated against BP Active Directory




Welcome email sent

18


with a picture in

the vertical-stripe

format, then adjust


2. User & usage reporting


2: User & usage reporting

Detailed reporting of ER/Studio and Repository usage for user tracking (and chargeback)Custom solutionDatabase of users

•User department & contact details•MAC address•Repository id

Licence server usage •Peak number of concurrent users (are we approaching licence

limit?)•Number of unique users registered and using service (monitor

take-up)


User & usage reporting

Concurrent License Usage

0

5

10

15

20

25

30

35

08

Ma

y

22

Ma

y

05

Ju

n

19

Ju

n

03

Ju

l

17

Ju

l

31

Ju

l

14

Au

g

28

Au

g

11

Se

p

25

Se

p

09

Oct

23

Oct

06

No

v

20

No

v

04

De

c

18

De

c

01

Ja

n

Max UsageUnique Users

Log files are copied from the server and parsedUsage graph shows peak concurrent license usage and number of unique users for a given dayAllows license purchasing decisions to be based on actual usageAllows Data Modelling Environment take-up to be monitored

21


with a picture in

the vertical-stripe

format, then adjust


3. Automated publishing of models to SharePoint


3: Model Publishing - highlights

Publishing of models from repository to Data Modelling Environment SharePoint

•Completely automatic generation of models in HTML (no need to produce ER/Studio report settings files)

Usual approach is to utilise report wizardApproach would be unworkable for organisation’s large # of

modelsAutomatically generate report settings files

•Customise generated reportsLayouts, title etc

•Automatic uploading to SharePoint Uploading of 1000’s of files to SharePoint is very problematic Restart built into our upload jobs

•Report home page in SharePoint mimics repository structureHighlights when repository models and SharePoint reports not

synchronisedPublishing meta data to inform users of status


Issues faced

1. How to view models without client reader

2. How to upload models to SharePoint

3. Too many files created in HTML publication

4. Size of model images

5. How to directly link to sub-models


1. How to view models without client reader

Inspired by the Watkins Motors Publishing approach we looked at how we could build upon this

The WM approach still had several manual steps

The data models are constantly changing, so the admin overhead associated with manual publishing steps would be too high – we didn’t want to have ANY manual steps.

SharePoint – chosen solution.


2. How to upload models to SharePoint

Investigated 3 options:

1. Shell automation

2. Web DAV

3. Vermeer RPC protocol


How to upload to SharePointOption 1: Shell automation•Message boxes were always popping up. •No “retry” method available•“Suppress errors” is not supported in Shell Automation.

Option 2: Web DAVA protocol for sending files & versions to SharePoints.But, internal internet security appliance blocks this protocol & the red-tape to get this changed would be enormous.

Option 3: Vermeer RPC protocolWeb folders use this. Standard within MS FrontPage (VTI folders)We investigated an example / sample implementation & found it’s written in .NETWe re-wrote this in VB6 for Excel


3. Too many files created in HTML publication

• For example, a single sub model had 550+ files generated;

• All bar 24 were empty / dummy files.

• We wrote routine to identify & then delete these before running the SharePoint publishing processes.


Model Publishing - Example

Query for updated models

Generate settings file

Generate HTML version of model

Upload HTML to SharePoint

Generate and update repository page

Repository

Repository Server




Firewall



Model publishing


4. Size of model images

Automatically generate report setting file

We didn’t want any manual steps in creating a report settings file

But by default zoom level of image is 100% - for lots of our large models this is too big & will result in image not being generated – why? Runs out of memory to render image

ER/Studio 7.1.1 allows specification of zoom levels for sub-models

We could now auto generate configuration settings files so that model images would be published ok


5. How to directly link to sub-modelsFor integrating with our process modelling tool

(ARIS) we want to be able to click on a link (in ARIS) to go directly to the SharePoint having the sub model.

Option 1: Parse full generated report & then re-generate HTML for the sub-model: Too slow - models too large

Option 2: Generate each individual sub-model on their own:

Option 3: Need to create tree of sub-models:

But too many redundant sub models generated by ER/S reporting;

So we added configuration parameters to specify the level of sub models we wanted publishing - business requirements only needed sub models at a particular level

33


with a picture in

the vertical-stripe

format, then adjust


Next steps & issuesFurther workflow steps in SharePoint

Interactive training – certify internal users

Expand Community of Interest

Web portal to interrogate repository

How to restrict licence server use to only approved users

Can’t “restrict” unregistered users who know licence server name (unless we build a proxy server)

Laptop change = new MAC address. Are they a legitimate user?

Internet access to repo & licence server

Linking model artefacts to drive re-use (e.g. Entities from Master Data Models)


A final thought

Did you miss the Web 2.0 & collaboration seminar?

Don’t worry – this has been Web 2.0 & collaboration in practice !

Weblog RSS

Libraries Survey

Intranetsites

Discussions

ListsWiki

editor

Workflow


Thank you

Contact details:Email: [email protected]: [email protected]: +44 (0)7973 184475MSN: [email protected]: www.ipl.com


Backup slides



Repository

Repository Server




Firewall






Generate and update repository pageSharePoint is queried to get publication dates and

compared with the last modified dates in the Repository



Repository

Repository Server




Firewall






Generate and update repository pageA Report Settings File (XML) is generated from the

model structure



Repository

Repository Server




Firewall







ER/Studio’s built in facility is used to generate the HTML reports



Repository

Repository Server




Firewall







HTML files are customised then uploaded to the SharePoint. Upload is unreliable so multiple retry

loops had to be implemented



Repository

Repository Server




Firewall






Generate and update repository pageRepository page is generated from the Repository

structure, shows when model changed and if model is out of date


In the spreadsheet we say PF_LDM_FINAL should publish sub-models at level 1 only (level 0 is the top level). We actually have a config parameter called "Publish Submodels" and it is set to "PF_LDM_FINAL.dm1(1-1);GOIL_LDM(1-1).dm1".

data modelling as a service in a super major

Documents