custom star creation for ellucain's enterprise data warehouse

PowerPoint Presentation

Creating Scalable Custom Stars for Ellucians EDW Bryan L. Mack04-03-2017

1

What is this presentation?This is a step-by-step high-level, yet technical guide discussing how to plug a new star schema into Ellucians EDW software.

This presentation can be used to model a new star, design a new star, and plug in a new star.

If you arent technical, this isnt for you. This presentation also does not discuss the business requirements; this is simply the technical side which was created to support the business requirements.

What were trying to do hereDevelop a new snapshot analytical star for Human resources which will store payroll benefit deduction data

Star will be made visible by way of:New fact table (ZFT_*)New dimension table (ZDT_*)Other dimension tables used in other pre-existing stars

Note: Our naming convention is to use Z for the first character for all custom database objects

So Why is this considered scalable?Not modifying any delivered objects*Using some delivered dimensions while adding custom dimensionsCan use the initial design from one star to develop future starsDatabase PackageMappingsTablesEtc.

*if you choose to name custom objects as Z*, a slight modification to a procedure is necessary; if you begin objects with W*, no objects need to be modified.

Deduction Star PurposeThe Employee & Employee Position stars aggregate all deductions but do not allow for reporting of:Aggregates of specific deductions over timeAggregates of specific types of deductionsHealth Insurance deductionsLife Insurance deductionsTaxesEtc.Aggregates of specific carriers over time (ie: Kaiser vs. BCBS)

Requirements Daunting!New Tables (Input/clean/wkeys/fact/dimension)Indexes & KeysNew Sequence for DIM tableTrigger for sequence on DIM tableNew Extract package (or table function)New cleansing rules & data elementNew mappings (input/clean/wkeys/fact/each dimension)New Cognos model (or modification to existing model)

.and dont forget your synonyms and grants!

Deduction Star DesignFact:ZFT_EMPLOYEE_DEDUCTIONDimensionsWDT_MULTI_SOURCEWDT_TIMEWDT_ADMINISTRATIONWDT_DEMOGRAPHICWDT_EMPLOYEEWDT_PERSONZDT_DEDUCTION --- CUSTOM!

Deduction Star Design Fact Measures:Age (non-additive)Years of Service (non-additive)Employee Deduction AmountEmployer Deduction Amount

Dimension Attributes:Carrier Code (and description)Coverage Option (and description)Deduction Category (Health, Life, Dental, Tax, etc.)



GitHubDue to the length of the code, I will only include screen shots of some blurbs of code. For my full source code & mapping dump, visit my GitHub page:

https://github.com/fleetmack/personal/tree/master/hr_deduction_star

Note: Every piece of code in that Git repo was written by me; I have not included a single piece of code provided by Ellucian as that is not my intellectual property.

Input TableName: ZTT_EMPLOYEE_DEDN_INPUTenable NoLogging within OracleWhat it is: Temp table before we clean & pivot the dataFields: Every field need to populate each of the dimension & fact tables

.

Clean TableName: ZTT_EMPLOYEE_DEDN_CLEANWhat it is: pulls in standardized descriptions that must conform to pre-defined cleansing rules, used to populate facts & dims

Fields: Everything in INPUT table Descriptions of all attributes (sd, ld, etc.)

Wkeys TableName: ZTT_EMPLOYEE_DEDN_WKEYSWhat it is: Extract of the data to be ported to the fact tableFields: Everything that goes into the fact table:

Dimension TableName: ZDT_DEDUCTION

Fact TableName: ZFT_EMPLOYEE_DEDUCTION

Note: My source code has a final column, mif_value_vc; this is a virtual column populated by a separate function, which Ive included in my GIT repo. This is custom to our institution as we use a virtual private database, it is not applicable to most schools



Sequence for Dimension PKThis will be used to populate ZDT_DEDUCTION.DEDUCTION_KEY



PK TriggerWe will use a trigger to insert the deduction_key to the dimension table:

Note: source code for this is within the zdt_deduction table DDL in my Git repo



Wait, what? You skipped several steps!Dont judge me, well get back on track after these synonyms & grants!

SynonymsAside from obvious reasons, your OWB/ODI CLEAN mappings will fail without these:

GrantsNeed these grants for all tables (input, clean, wkeys, dim, fact) in order for your mappings to work:

grant ALTERon ztt_employee_deduction_clean to IA_ADMIN;grant DELETEon ztt_employee_deduction_clean to IA_ADMIN;grant INDEXon ztt_employee_deduction_clean to IA_ADMIN;grant INSERTon ztt_employee_deduction_clean to IA_ADMIN;grant SELECTon ztt_employee_deduction_clean to EDWMGR;grant SELECTon ztt_employee_deduction_clean to IA_ADMIN;grant UPDATEon ztt_employee_deduction_clean to IA_ADMIN;grant REFERENCESon ztt_employee_deduction_clean to IA_ADMIN;grant ON COMMIT REFRESHon ztt_employee_deduction_clean to IA_ADMIN;grant QUERY REWRITEon ztt_employee_deduction_clean to IA_ADMIN;grant DEBUGon ztt_employee_deduction_clean to IA_ADMIN;grant FLASHBACKon ztt_employee_deduction_clean to IA_ADMIN;



Extract PackageName: EDWSTG.zdw_employee_dedn_extr

Extract PackageContains:Table FunctionPopulation of all values for dimensionsPopulation of all values for fact table

Package code on GitHub



New Cleansing Rule for Dimensional dataAdmin UI tool -> Options -> Set Up and Maintain Cleansing Processes -> Set up and Maintain Cleansing Rules

Note: Youll need to do this for all detail record in your dimensions

New Cleansing Rule for Dimensional Data

New Data ElementAdmin UI tool -> Options -> Set Up and Maintain Cleansing Processes -> Set Up and Maintain Cleansing Data Elements

Associate the fields in your dimension with the proper cleansing rule:



Build & Deploy Input MappingInput Parameters call your table function:

Table Function populates the Input table:

Build & Deploy Input MappingEnsure you set the Input table to truncate/insert so that you start fresh each time you snap the data:

Build & Deploy Clean Mapping

Build & Deploy Clean MappingP_CLEANSE_INPUT procedure will do most of the work for you:

Constant.Tablename: EMPLOYEE_DEDNConstant.Source: OP_CELANSE_INPUT source: MGKFUNC.P_CLEANSE_INPUT

Modify MGKDCLS.P_CLEANSE_INPUTRemember when I said we may need to modify 1 piece of delivered code? This is ONLY if you choose to name your custom objects beginning with Z ..

Build & Deploy Dimension Mappings

Dimension ETL Mapping Info TidbitsTarget dimension table (ex: WDT_ADMINISTRATION) needs its PK (ex: administration_key) to have its loading properties set as such:

Dimension ETL Mapping Info Tidbits

Dimension ETL Mapping Info TidbitsTarget table (ex: WDT_ADMINISTRATION) codes (exs: employer_code, home_organization, etc.) need to have loading properties set as such:

Target table descriptions (exs: employer_code_sd, employer_code_ld) have loading properties as such:

Dimension ETL Mapping Info TidbitsTarget table (ex: WDT_ADMINISTRATION) needs its system_load_tmstmp loading properties set as such:

Dimension ETL Mapping Info Tidbits

Dimension ETL Mapping Info TidbitsTarget table properties of note

Building & Deploying Dim MappingsRepeat this process for all dimensions within the star

Build & Deploy WKEYS mappingUse the clean & dimension tables to build this temp-table. The WKEYS table will be used to populate the fact.

WKEYS ETL Mapping Info TidbitsAll of your keys should have loading properties as such:

WKEYS ETL Mapping Info TidbitsYour measures should have loading properties as such:

WKEYS ETL Mapping Info TidbitsNo Contraints:

Truncate/insert

Fact Delete ETL MappingUsed in case you are replacing a particular Snapshot, will delete the old rows before you replace them.

Fact Delete ETL MappingJoiner Condition:

This Y is an input parameter when scheduling the star. Are you replacing data, or not? If you are not replacing data, this mapping will be skipped.

Fact ETL Mapping Info TidbitsKey Loading Properties:

Measure Loading Properties:

Fact Insert ETL Mapping

Key Loading Properties: Yes, No, Yes, YesMeasure Loading Properties: Yes, Yes, No, No

Our Mappings

MDL file with mappings and all dependent objects are stored on GitHub

How will we populate the star?Lets modify some parameters in our MTVPARM table by way of Ellucians Administrative utility (Admin UI) to allow us to run these jobs through the GUI .

Admin UI Multi Source prompt

Admin UI Event prompt

Admin UI Calendar Year prompt

Admin UI Calendar Month prompt

Admin UI Replace IND prompt

Admin UI Create ETL Map Package

Add Individual Dimensions to ETL Map Package

Add Wkeys/Fact Mapping to ETL Map Package

Admin UI Create ETL Control Group

Add Dimensions Group to ETL Control GroupAdd a second internal code group to handle all the dimensions for a wide load of the dimensions

Add Wkeys/Fact Group to ETL Control Group

Admin UI Create Subprocess

Ta Da!

Viewing Control Report

.

Data Populated!We now have data in our stars tables.

We still need to build a custom Cognos model & data descriptions for our users to utilize self-service reporting, but that will be covered in another presentation coming soon .

[email protected]

custom star creation for ellucain's enterprise data warehouse

Technology