2011 central tug tawfik tbio

8/2/2019 2011 Central TUG Tawfik TBIO

1/26

2011 Spring TUG

Sam TawfikProduct Marketing Manager

[email protected] @teradata_sam

Teradata Business IntelligenceOptimizer
mailto:[email protected]:[email protected]


2/26

Agenda

OLAP MOLAP/ROLAP Teradata Aggregate Designer

Teradata OLAP Connector Demo

Implementation Considerations


3/26

OLAP Environments

Marketing Customer ServiceFinance/Planning

Physical Cubes(MOLAP)



Multiple BIEnvironments

Cubes everywhere

Data replication

Long ETL jobs

Limited scalability

Limited data sets

Mid-tier server costs


4/26

Convert to ROLAP EnvironmentsEliminate Physical Cubes






5/26

ROLAP Solution


BI Server BI Server BI Server




Multiple BIEnvironments

Single View

Eliminate delays

DW scalability

Rich detailed data

Reduce/Eliminate need

for Mid-tier server


6/26

ROLAP Solution

Solution Convert MOLAP to ROLAP with Teradata Database

> Let Teradata host and analyze the data

> Use existing Teradata Indexes to optimize response time

> Use Teradata Aggregate Join Index (AJI) the Teradata

built-in OLAP optimization capabilities

Benefits High performance analytics powered by Teradata BI solutions benefit from the following:

> Increased breadth/depth of business information> Eliminate data latency> Eliminate/reduce need for the mid-tier server> Avoid mid-tier scalability and performance limitations


7/26

OLAP Implementations Attributes

Attribute MOLAP HOLAP ROLAP

Flexibility Application Specific Application Specific Application Neutral

Query

Performance

Sub-secondsSub-seconds for cube;few seconds for detail

Few seconds

Breadth ofAnalytics

Limited to Cube onserver

Almost unlimited Almost Unlimited

Depth ofAnalytics

Limited to Cube onserver and drillthrough

Drill-down for detailedanalysis

Detailed analysis

Data Freshness Varies depending oncube refresh

Combination of cubedata and current

Current data

Cost ofImplementation

High Moderate Low


8/26

Customer Example

38 dimensions and 24 measures with twoyears of history

> Added 39th dimension (Wire Center)

MOLAP solution couldnt handle thebusiness change

Implemented ROLAP solution using AJIs

> History expanded from 24 to 38 months

> # of dimensions expanded from 38 to 50

> # of measures expanded from 15 to 21

Results

> Maintenance: 13 hoursto3 minutes

> Cube size: 22.4 GB to Detail: Month to Daily

ROLAP Implementation

0

50

100

150

200

250

300

5 Canned 5 Canned with Wire

Center

6 Interactive 6 Interactive with Wire

Center

MOLAP

ROLAP with AJIs

Response Comparison:


9/26

Teradata Aggregate Designer


10/26

AJIs IntroducedV2R3.0.2

Sparse JI/AJI, Single AMP, Group-AMPV2R5.0

RIV2R5.1

PPIV2R6.2

M-PPI12

PGB, Cost-based re-writeOUTER JOIN, COUNT,

CUBE, ROLLUP, GROUPING SET,Subqueries, Spool derived table

13

Teradata Aggregate Join Index Timeline


11/26

Aggregate Join IndexPre-aggregated Results and Calculations

For frequently needed roll-up summaries> Sales results, orders, shipments, etc.

> Bypass join, sort, sum at run time

> Applies for lower level aggregations (many transactions)

Good for

> Dashboards, portal

> Interactive analytics

pre-aggregatedresults

Fast responsetime from using

aggregated results


12/26

ROLAP Implementation with Teradata

1. Identify the target solution start with a small cube2. Capture the cube schema definition

3. Validate the BI schema and physical data model are suitable for ROLAP

4. Perform data quality checks

5. Create the semantic data model

6. Select the target database for the ROLAP solution

7. Select the aggregation levels for each dimension

8. Create the necessary base and broad AJI(s)

9. Review the Aggregate Join Index DDLs and modify as necessary

10. Validate/Publish the AJIs on the target database

11. Perform one data loading job from source to the fact table

12. Test the AJIs to make sure they are being used by the Optimizer13. Collect statistics

14. Tune the AJIs as the BI solution evolves

15. Gradually add more features to the cube or create new cubes


13/26

Description> Desktop visual administrative tool to automate designing,

building, and managing AJIs

> Natively import cube definitions from IBM Cognos, Microsoft SQLServer Analysis Services, or Teradata Schema Workbench

Release 13.10 Enhancements

> Native support for IBM CognosFramework Manager(8.3, 8.4, and 10)

> Ease of use and functionalityenhancements

> Enhanced database and schema validations

> Detect and alert Schema changes

> Support AJI enhancement in 13.10

Teradata Aggregate Designer


14/26

Teradata Aggregate Designer Features

Database Validation> Validates target database schema

for AJI requirements

> Recommends possible databaseschema changes

AJI Advisor

> Uses Teradata-specific heuristics torecommend the optimal AJI design

AJI Creation Services

> Provides AJI storage cost estimates

> User can manually modify AJI levels

> Validates the SQL and creates

Teradata DDL

Supports PI and PPI definitions


15/26

Teradata Aggregate Designer automatesdesigning, building, and managing AJIs for ROLAPsolutions

Semantic Layer

Office SharePoint Server

SQL ServerAnalysis Services

Office

Performance

PointServer

SQLServer

Reporting

Services

Teradata .NET Data Provider

Excel

PivotTables

TeradataAggregate Designer

AJIsAJIsAJIs

Workflow

1. DBA leverages the Teradata Aggregate Designer

tool to consume cube schema from SQL ServerAnalysis Services

2. DBA creates/modifies necessary AJIs usingTeradata Aggregate Designer

Optimized SQL

Microsoft BI with Teradata ROLAPLeveraging Teradata Aggregate Designer


16/26

Portal

MetaData

Scorecard

Cognos Server

Reporting

Dashboard

Workflow

1. BI Admin creates and manages OLAP schema in

the Cognos Framework Manager.2. Teradata Aggregate Designer consumes model.xml.

3. Teradata Aggregate Designer validates database issuitable for Aggregate Join Indexes. Recommendsand creates AJIs.

Teradata Aggregate Designer automates thedesign and deployment of AJIs for BI Reporting,Dashboards, ROLAP, and MOLAP solutions

IBM Cognos with Teradata ROLAPLeveraging Teradata Aggregate Designer

TeradataAggregate Designer

AJIsAJIsAJIs

Optimized SQL


17/26

Teradata Aggregate Designer Demo


18/26

Teradata OLAP ConnectorExcel to Teradata Connectivity

Teradata OLAPConnector

MDX

SQL

1. Open a new Excel workbook

2. Establish data connection to Teradata Database via Teradata OLAP Connector

3. Select the project (catalog), and then select the cube

4. Use Excels PivotTables to navigate/view the data Teradata OLAP Connector convertsPivotTable requests into optimized Teradata SQL


19/26

Teradata OLAP Connector Demo


20/26

Implementation Considerations


21/26

Implementation Approach

Moving from MOLAP to ROLAP requires planningand changes to the BI application

Customers should balance the work to be doneto migrate from MOLAP against theadded benefits of ROLAP

Recommendations

1. Start with a ROLAP PoC to validate the approach

Typically the PoC is completed in a short period of time and with asmall budget so make sure you select an appropriate cube to model

The PoC should focus on the ROLAP solution itself and not the BIchanges

Consider using Excel PivoTables to demonstrate the navigation andreporting of the ROLAP implementation

2. Once the ROLAP PoC has been completed then plan for widerimplementation


22/26

Data Loading Considerations

Teradata Multiload does not support AJIs, following arethe different load scenarios to consider:

> Multiload into normalized tables, and then build AJIs fromthe normalized tables

> Teradata Tpump into normalized tables

> Teradata Fastload into staging tables, and then insert selectinto the base tables


23/26

Teradata Database Considerations (1 of 2)

We recommend you start with star and snowflake models All primary and foreign keys are defined as not nullable

All primary and foreign keys are not compressible.

All dimension table primary keys are defined as unique

Ensure all primary and foreign keys are on ID not Name or

Description columns. This will result in a smaller AJI whichmeans faster access.

Ensure all measures in the Fact table are set with Floatdatatype. Otherwise, overflow may occur on some largecalculated measures

Teradata Release 12 or later


24/26

Teradata Database Considerations (2 of 2)

Recommended Fact Table design is a 'wide' (i.e. columns for eachdimension and measure)

Single level dimensions need supporting reference/lookup/dimensiontable for optimal performance

Collect statistics on all primary key/foreign key relationship columns

Implement Referential Integrity (RI) on the primary key/ foreign keycolumns. RI can be defined with the check (a.k.a., hard RI) or nocheck option (a.k.a., soft RI).

> ORGS: Business Unit -> Division -> Area -> Sales Center

> ALTER TABLE FACT ADD foreign key (SALES, CENTER_ID) references withno check option SALE_CENTER (SALES_CENTER_ID)

To address the lowest levels in your analytics (i.e. your FACT) thatare not in your AJI, consider the following:

> Secondary Indexes

> Partition Primary Index


25/26

Performance and Capacity Assessmentsand Services

Data Collection and Reporting Collect

and analyze key performance metrics Workload Management Deliver maximum

throughput of mixed workloads based

on query mix and business priorities

Application Performance Identify target

applications for performance improvements andtune those applications

Performance Assessment Quick hit service desinged to analyze andidentify opportunities for performance improvements

Capacity Management Understanding growth requirements andfuture needs based on application growth, performance requirements,

and user sophistication Viewpoint Advisor Service Best practices implementation of

Viewpoint Administration and Portlet use

Performance Management Foundation Foundational ServiceDesigned around Making the Customers Performance Team selfsufficient in all these areas

CapacityManagement

DataCollection


26/26

2011 central tug tawfik tbio

Documents